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A  model  for  investigating  alternative  research  and  development  programs  is 
formulated.  Cost-constrained  optimization  methods  are  used  in  an  expected  value 
formulation  for  systems  which  have  reached  the  concept  development  stage.  For 
the  remaining  projects,  those  in  basic  research  and  exploratory  development,  decision 
rules  for  altering  funding  levels  are  suggested.  The  principal  variables  considered  in 
the  model  are  (1)  the  relative  value  of  a  system,  a  subjective  value  judgement,  (2) 
the  expected  life  of  the  system,  (S)  decision  maker  time  preference,  and  (4)  the 
cost  of  the  system.  A  management  information  system  for  implementing  the  model 
is  proposed  which  allows  the  user  to  focus  on  the  trade-off  implications  of  any  ol 
the  alternatives  available  for  ntodifying  the  budget  of  the  R  &  D  program.  The 
specific  decision  problems  the  model  addresses  are  those  faced  by  the  R  &  D'^lanners 
of  the  U.  S.  Army.  However,  the  model  could  also  be  applied  to  industrial  research 
and  development  programs.  No  extensive  knowledge  of  mathematics  is  required  of 
the  reader;  however,  explanations  of  various  mathematical  concepts  are  discussed 
in  an  appendix. 
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ABSTRACT 


A  model  for  investiga!'  ^  .(ic  native  research  and  development  programs  is 
formulated.  Cost-constrained  optimization  methods  are  used  in  an  expected  value 
formulation  for  systems  which  have  reached  the  concept  development  stage.  For 
the  remaining  projects,  those  in  basic  research  and  exploratory  development,  decision 
rules  for  altering  funding  levels  are  suggested.  The  principal  variables  considered  in 
the  model  are  (1)  the  relative  value  of  a  system,  a  subjective  value  judgement, 

(2)  the  expected  life  of  the  system,  (3)  decision  maker  time  preference,  and  (4)  the 
cost  of  the  system.  A  management  information  system  for  implementing  the  model 
is  proposed  which  allows  the  user  to  focus  on  the  trade-off  implications  of  any  of 
the  alternatives  available  for  modifying  the  budget  of  the  R  &  D  program.  The 
specific  decision  problems  the  model  addresses  are  those  faced  by  the  R&D  planners 
of  the  U.  S.  Army.  However,  the  model  could  also  be  applied  to  industrial 
research  and  development  programs.  No  extensive  knowledge  of  mathematics  is 
required  of  the  reader;  however,  explanations  of  various  matehmatical  concepts 
are  discussed  in  an  appendix. 
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L  INTRODUCTION 


The  information  processing  capacity  of  any  one  man  is  limited.  Faced  with 
potentially  enormous  quantities  of  data  pertaining  to  a  decision  problem,  the 
task  of  the  decision  maker  is  to  select  the  significant  information  from  the  trivial 
and  to  identify  an  appropriate  course  of  action.  However,  even  the  identification 
phase  of  this  process  is  a  difficult  task  when  the  problem  is  as  large  as  that  of 
budgeting  the  research  and  development  program  of  the  U.  S.  Army.  Narrowing 
the  scope  of  the  problem  to  one  of  selecting  a  budget  for  a  particular  system  under 
development  makes  the  problem  more  manageable,  but  the  impact  of  this  selection 
on  the  program  as  a  whole  may  not  be  veadily  visible.  With  limited  resources  a 
decision  on  a  budget  for  one  system  necessarily  affects  all  the  others.  The  decision 
problem  then  becomes  one  of  identifying  what  should  be  sacrificed  if  any  .system 
is  to  be  increased.  In  an  R  &  O  program  as  large  as  that  of  the  Army’s,  the  alterna¬ 
tives  are  almost  countless  and  the  time  available  for  considering  the  problem  is 
limited. 

The  Computer  Assisted  Research  and  Development  Budget  Optimization  Model 
(CARDBOMB)  is  a  cost-constrained,  value  maximization  model  which  considers 
the  interactions  of  the  budgets  of  all  the  projects  in  the  R  &  D  prograni.  It  adds 
insight  to  the  question;  “Based  on  the  information  gathered  during  the  planning 
phase  concerning  the  value  and  cost  of  each  proposed  project,  how  should  X 
billion  dollars  be  distributed  among  these  projects?’’ 

For  development  programs  which  are  at  or  b;;yond  the  concept  formulation 
stage  of  development,  the  model  uses  optimization  techniques  for  providing  insight 
into  this  question.  The  variables  considered  in  the  model  are: 
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1.  The  value  of  a  system  (the  end  product  cf  a  developmer  plan);  a  function 
of  tioio,  need,  useful  life,  uncertainty  and  dollar  costs. 

2.  The  cost  of  a  system;  a  function  of  development,  procurement  and  oper* 
ating  costs,  performance  characteristics  and  operational  readiness  dates, 
time  and  uncertainty. 

The  model  also  deals  with  basic  research  and  exploratory  development  projects, 
i.e.,  those  not  diiectly  associated  with  a  program  which  has  reached  the  concept 
development  stage.  Optimization  methods  are  not  used  for  these  projects,  but 
decision  rules  are  suggested  for  investigating  alternative  budgets. 

Methods  for  quantifying  the  value  of  a  system  are  discussed  at  length.  Although 
tiie  meaning  of  the  terms  “value”  and  “system”  are  dealt  with  more  explicitly 
later,  it  jhould  prove  useful  to  introduce  them  here.  An  R  &  D  system  is  the  end 
product  that  is  expected  to  result  from  one  or  more  projects,  e.g.,  the  ^oal  of  a 
system’s  development  plan.  In  ti  e  hardware  area  the  Cheyenne  helicopter  or  the 
MBT  70  tank  arc  examples  of  systems.  In  a  non-hardware  area  such  as  research  in 
human  performance,  a  system  is  the  specific  knowledge  that  is  expected  to  be  gained 
from  one  or  more  research  projects.  For  example  a  system  in  the  category  of 
human  performance  might  be  termed  “extending  the  endurance  limits  of  the  individual 
soldier,”  consisting  of  a  number  of  research  or  test  projects.  Systems  are  then 
categorized  by  general  type,  e.g.,  air  mobility,  missiles,  human  performance,  etc. 

Within  these  categories  experts  are  asked  to  use  their  own  subjective  value  judgement 
in  rating  the  relative  importance  they  place  on  the  systems  ii;  their  category.  Methods 
are  then  devised  for  transforming  the  interval  scale  of  measurement  scores  of  each 
category  onto  an  int-'rval  scale  for  the  entire  R  &  D  program  so  that  the  value  of  a 
system  in  one  category  can  be  related  to  the  value  of  a  system  in  aiiother.  Thus 
value,  rather  than  being  some  intrinsic  pro|)erty  of  a  system,  is  c  po  ‘.'ptive  notion 
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derived  by  considering  what  the  system  is  designed  to  accomplish,  and  how  much  this 
capability  is  required. 

A.  FORMULATION  OF  THE  RESEARCH  AND  DEVELOPMENT  PROGRAM 

Before  beginning  the  development  of  the  model,  it  should  prove  useful  to 
briefly  discuss  how  Program  6.  Research.  Development.  Test  and  Evaluation  (RDTE), 
of  the  Army  budget  is  formulated.^ 

Requirements  for  new  weapon  systems,  and  therefore  R  &  D  projects,  are 
identified  by  a  number  of  organt2ations.  First,  the  intelligence  community  makes 
known  the  current  and  projected  capabilities  of  potential  enemies.  Second,  the 
studies  conducted  by  the  Combat  Developments  Command  and  other  agencies  con¬ 
cerning  the  organization  and  tactical  employment  of  the  Army  uf  the  future  identify 
areas  in  which  weapons  technology  must  be  advanced.  Additionally,  technologies 
advances  have  their  own  way  of  creatii  "  needs  for  new  systems.  For  example, 
a  breakthrough  in  research  on  tank  engjies  may  lead  to  the  decision  that  the  most 
cost-effective  step  that  can  be  taken  is  to  develop  a  new  tank.  This  ties  in  closely 
with  a  fourth  way  that  ne  ds  arise,  technological  obsolescence.  Some  systems, 
designed  with  the  technology  of  the  past,  have  lost  much  of  their  initial  value  due 
to  counter  developments  by  potential  enemies. 

AU  of  these  needs,  or  requirements,  are  channeled  into  the  Army’s  in-house 
R  &  D  community  for  validation  of  the  need  and  investigation  of  possible  alter¬ 
natives.  This  proce.ss,  which  includes  extensive  inve.stigation  of  the  new  technologies 
that  would  have  to  be  developed  to  support  the  v'arious  alternatives,  is  called  vlic 
concept  formulation  stage.  Finally  the  point  is  reached  where  a  development  plan 

^For  a  more  detailed  discussion  of  the  .Army’s  R  &  D  organiz;;  n  and  procedures, 
as  well  as  the  overall  Department  of  Defense  management  of  K  .lit  D,  >ee  Sanders  [ref.  IJ. 
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item  projects  in  Program  6  of  the  Five  Year  Defense  Plan  (FYDP),  a  planning  and 
budgeting  document  maintained  by  the  Army’s  Chief  of  Research  and  Development 
(CRD).  It  should  be  noted  at  this  point  that  not  all  of  the  more  than  600  line  item 
projects  in  Program  6  are  directly  related  to  a  specific  system.  Many  of  them  per¬ 
tain  to  overhead  of  the  R  &  D  in-house  community  (Sub-Program  6.5  —  Management 
and  Support)  as  well  as  basic  research  (Sub-I’rogram  6.1)  not  directly  associated 
with  a  particular  system.^  Program  6,  like  all  the  Army  budget  programs,  might 
be  called  a  living  document.  As  was  mentioned,  new  projects  are  added  every  time 
a  new  DCP  is  approved.  Additionally,  within  the  bounds  of  the  threshold  points, 

DA  may  often  alter  the  funding  levels  of  various  projects.  However  at  some  point 
in  time  during  the  annual  budget  cycle,  the  CRD  submits  the  particular  Program  6 
that  he  recommends  for  inclusion  in  the  Army  budget  to  be  submitted  to  DOD  for 
approval  and  forwarding  to  the  President.  For  tne  purposes  of  this  paper,  this  five 
year  funding  schedule  for  all  Program  6  projects  and  the  funding  schedules  for  each 
of  them  in  ihe  years  beyond  the  FYDP  up  to  the  last  year  in  which  RDTE  funds 
are  planned  will  be  called  the  base  case. 

B.  THE  PROBLEM:  INVESTIGATING  ALTERNATIVE  BUDGETS 

Once  a  Program  6  base  case  has  been  recommended  by  the  Chief  of  Research 
and  Development,  the  final  decision  on  the  program  to  be  recommended  to  the 
President  is  far  fro..i  reached.  The  Army’s  Budget  Review  Committee  (BRC)  and 
otlier  top-level  committees  as  the  Secretary  of  the  Army  of  the  Chief  or  Staff  may 
direct  must  then  investigate  how  this  base  case  fits  in  with  other  Army  programs, 
and  what  adjustments  might  or  must  be  made  to  insure  compliatice  with  national 

^More  detail  concerning  ihe  conteui,  of  Program  6  will  be  pres  led  in  Section  II, 

C,  “A  Ta.xonomy  of  R  &  D  Systems.” 
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security  objectives  and  the  DOD  fiscal  guidance  constraints.  This  is  not  to  imply 
that  these  considerations  are  disregarded  in  aniving  at  the  Program  6  base  case.  On 
the  contrary,  it  was  formulated  throughout  the  year  in  an  iterative  fashion  with  just 
these  considerations  in  mind.  However,  during  the  “budget  crunch”  months  of 
October  through  December,  difficult  trade*off  decI:dons  must  be  made,  and  the  BRC 
plays  an  increasingly  important  role.^  In  essence  they  must  investigate  alternative 
mixes  between  Army  programs  which  in  turn  determine  the  level,  or  total  bucigit 
constraint,  of  each.  To  do  this  with  as  much  insight  as  possible  in  order  to  avoid 
setting  arbitrary  limits  on  the  levels  of  programs,  it  is  also  necessary  for  them  to 
investigate  mixes  within  the  programs.  Unfortunately  the  time  constraints  of  the 
PPBS  cycle  may  limit  this  investigation  substantially.  It  is  not  inconceivable  to 
imagine  that  only  the  highest  cost  or  critically  important  items  get  carefully  investi¬ 
gated,  while  other  projects  in  a  program  might  have  to  experience  something 
approaching  an  “across  the  board”  percentage  change,  usually  a  reduction.  While 
no  empirical  evidence  is  offered  here  to  support  this  contention,  it  would  appear 
logical  that  the  more  time  available  for  identifying  and  investigating  alternatives, 
tlie  better  the  decision. 

Ultimately  a  budget  gets  approved  by  the  Secretary  of  the  Army  for  forwarding  to  DOD, 
but  the  budgeting  problem  is  still  far  from  resolved.  DOD  continues  to  n.ake  Program  Budget 
Decisions  which  are  either  accepted  or  rebutted  by  the  Services.  Additionally,  either  because 
the  Office  of  Management  and  Budget  has  altered  the  budget  constraint  of  DOD  as  a  whole 
or  because  DOD  wishes  to  investigate  the  alternative  mixes  between  the  Services,  the 

^jrmy  may  be  directed  to  submit  n  new  program  constrained  at  some  different  level. 

The  sometimes  severe  time  constraints  on  these  important  deliberations  and 

the  need  for  vast  quantities  of  readily  available  information  pertaining  to  individual 

^Although  the  BRC  is  not  the  only  agency  making  rccommon  ions  to  the  SA 
and  COS  concerning  the  budget,  for  simplicity  this  pai)er  will  refer  ly  to  tlu  m  as 
the  principal  advisors. 
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programs  has  led  the  Office  of  the  Assistant  Vice  Chief  of  Staff,  Aimy  (AVCSA) 
the  Army’s  coordintiting  office  for  matters  pertaining  to  the  PPBS,  to  move  toward 
automated  management  information  systems.  This  paper  is  directed  at  assisting  in 
this  effort  within  the  context  of  the  existing  decision  problem  and  organizational 
structure.  Suc  specific  decision  problems  representing  the  range  of  options  for 
altering  the  Program  6  budget  are  addressed.  These  are: 

1.  Should  the  funding  of  a  system  be  reduced? 

2.  Should  the  funding  of  a  system  be  increased? 

3.  Should  the  schedule  of  a  system  be  slipped,  freeing  funds  for  a  particular 
year?  (Slipping  is  defined  as  delaying  the  start  of  a  proposed  development 
program,  or  cutting  it  off  somewhere  in  mid-cycle  with  the  intent  of 
continuing  it  at  a  later  date.) 

4.  Should  a  ntiw  system  just  finishing  the  concept  development  stage  be 
added  to  tiie  program? 

5.  Should  a  sj'btem  be  dropped  from  the  piogram? 

6.  Should  funds  be  added  to  or  taken  away  from  the  total  RDTE  budget? 

The  extent  of  the  sb  decision  problems  addressed  by  CARDBOMB  shows  that  its 
potential  users  might  be  the  planning  staffs  of  the  Director  of  the  Army  Budget, 
the  BRC,  the  AVCSiV  or  the  CRD. 

C.  CONSIDERATIONS  IN  MODELING  RESEARCH  AND  DEVELOPMENT 

There  are  chara:teristics  of  R  &  D  programs,  both  military  and  industrial, 
that  make  them  particularly  difficult  to  manage.  A  brief  introduction  to  some  of 
these  problems  is  pn!sentcd  here  so  that  the  reader  will  be  cognizant  of  them  and 
follow  more  carefully  how  each  of  them  are  dealt  with  in  the  model.  Chief  among 
these  difficulties  is  uticertainty. 

The  uncertainty  of  the  need  for  a  particular  system,  sometime  called  scenario 
risk,  is  usually  the  most  uncertain  factor.  The  intelligence  commun.iy  might  forecast 
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that  by  1986  every  Russian  soldier  wi<I  be  equipped  with  a  man-packed  jet  propulsion 
unit  capable  of  moving  him  above  ground  at  30  knots  for  great  distances.  There 
may,  however,  be  a  considerable  question  as  to  the  validity  of  the  forecast.  Addition¬ 
ally  there  may  be  uncertainty  as  to  the  best  approach  to  counter  this  move.  Finally, 
no  one  can  state  for  sure  that  by  1986  the  Russians  will  be  a  potential  threat. 
Perhaps,  if  the  future  were  known,  the  need  might  be  greater  in  countering  a  gamma 
ray  gun  development  in  Albania. 

The  uncertainty  of  the  availability  date  and  ultimate  performance  characteristics 
of  a  weapon  system  undergoing  development  are  termed  technological  uncertainty. 
Given  the  varying  degrees  of  the  state  of  the  art  in  different  technologies,  it  may 
become  necessary  to  settle  for  less  than  the  system’s  performance  objectives  formu¬ 
lated  in  its  current  DCP.  On  the  other  hand  a  major  technological  breakthrough 
might  allow  the  acliievemenl  of  far  more  value  thon  is  currently  envisioned. 

Similarly,  the  technological  development  problems  might  take  so  long  to  resolve 
that,  by  tlie  system’s  availability  date,  it  has  already  been  overtaken  by  events. 

For  example,  some  counter  R  &  D  move  by  a  potential  enemy  either  now  or  in 
the  future  might  seriously  degrade  the  value  of  the  system  as  currently  perceived. 

This  introduces  the  notion,  to  be  explored  in  greater  depth  in  Chapter  2,  that  the 
value  of  a  completed  system  is  not  constant  over  time. 

The  third  significant  factor  is  cost  uncertainty.  Particularly  during  the  early 
concept  formulation  stage,  R  &  D  and  otlier  important  cost  estimates  might  be  far 
off  the  mark.  Even  as  development  of  the  system  continues,  PEMA  and  OMA  cost 
projections  still  may  prove  to  be  significantly  different  than  projected. 

^Procurement  of  Equipment  and  Missiles,  Army  (PEMA)  and  Ojieration  and 
Maintenance,  Army  (OMA)  are  two  additional  budget  programs.  Bo  ii  the  procure¬ 
ment  and  operation  and  maintenance  costs  are  costs  that  must  be  i  nsidcred  with 
RDTE  funds  in  the  "develop  or  do  not  develop”  type  decisions. 
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Examples  of  the  implications  of  these  uncertain  factors  illuminate  thbir  inter* 
dependence.  If  for  a  given  system,  RDTE  costs  have  been  estimated  too  low,  PEMA 
funds  for  this  system  and  others  might  have  to  be  transferred,  thus  lowering  the 
total,  overall  value  to  be  gained.  If  the  need  has  been  overstated  and  Russia  never 
fields  jet-propelled  soldiers,  then  the  value  of  our  jet-propelled  Russian  barrier  may 
be  zero,  not  to  mention  the  value  bst  from  other  systems  that  might  have  been. 
develo;<ed  with  the  same  funds. 

Another  difficulty  encountered  in  the  modeling  of  R  &  D  is  its  lack  of  a  unit  of 
measure;  i.e.,  it  is  meaningless  to  talk  about  dollars  per  unit  of  R  &  D.  What  type 
of  an  R  &  D  unit  of  output  could  be  defined  such  that  two  million  dollars  would 
buy  twice  as  many  units  as  one  million?  The  MARK  TWAIN,  a  Leontif  Input-Output 
model  currently  being  used  by  DA  in  studying  the  force  structure  budgeting  problem, 
has  as  a  necessary  input  the  cost  per  unit  for  Fx/ii-e  X.  Tha  EXECUTIVE 
GUIDANCE/DECISION  MODELS,  an  eutomated  management  information  system 
al.so  used  in  DA  for  studying  Uie  PEMA,  OMA,  manpower  and  force  structure 
budgets,  also  require  these  linear  cost  estimating  relationships.  The  activity  analysis 
structure  of  these  two  models  makes  them  inappropriate  for  R  &  D.  primarily 
because  of  this  lack  of  a  unit  of  measure  of  output. 

It  may  be  enlightening  in  this  introduction  to  compare  the  purposes  of  these 
two  models  and  the  one  proposed  in  this  paper.  The  MARK  TWAIN  is  primarily 
a  costing  model.  The  user  inputs  the  force  structure  he  is  investigating  and  the 
model  generates  its  cost.  THE  DEAN  MACHINE,  on  the  other  hand,  incorpiorates 
some  built-in,  parameterized  decision  rules  for  the  alteration  of  the  budgets  of 

^The  reader  acquainted  with  Army  budgeting  models  will  pro!  ly  recognize 
this  by  its  more  familiar  name,  THE  DE.\N  MACHINE,  after  its  pr;  ;pal  author, 

LTC  A.  .M.  R.  Dean. 


specific  line  items  in  the  budget  program  the  user  is  investigating.  For  example,  the 
purchase  of  a  2Vi-ton  truck  might  be  the  first  priority  for  reduction  in  the  PEMA 
budget,  but  only  a  five  percent  reduction  can  be  absorbed.  The  user  inputs  the 
total  budget  constraint  he  is  investigating  for  the  PEMA  program,  and  the  model 
uses  the  decision  rules  to  produce  a  new  program  constrained  at  this  level. 
CARDBOMB  has  a  similar  objective.  The  user  inputs  a  toi-al  RDTE  budget  constraint, 
and  then  based  on  one  of  a  number  of  specific  strategj''s  he  may  wish  to  employ, 
a  sirgle,  mathematically  optimal  alternative  is  presented  which  both  maximizes  the 
value  of  the  entire  program  and  meets  the  total  budget  constraint  specified.  Of 
course  as  has  been  stated,  “value”  will  be  defined  more  explicitly  later. 

Despite  the  problems  posed  by  uncertainty,  lack  of  a  unit  of  measure  and 
other  factors,  extensive  efforts  have  been  made  to  model  R  &  D,  both  in  the  military 
and  the  industrial  sectors.  The  primary  emphasis,  however,  has  been  on  project 
selection  in  the  general  field  called  capital  budgeting.  Project  selection  models  have 
as  their  primary  emphasis  the  decision  problem  faced  when  a  new  system  is  proposed. 
For  the  most  part  they  are  not  concerned  with  incremental  changes  to  the  bucigets 
of  existing  development  programs.  However  some  of  these  models  could  be  very 
useful  to  the  Army  and  are  discussed  further  in  the  text. 

There  are,  however,  at  least  two  models  which  to  some  degree  do  address  the 
problem  of  determining  optimal  incrcmenUd  budget  changes.  The  first  of  these 
was  developed  by  McGlauchlin  [ref.  4]  for  the  Honeywell  Corporation.  In  brief, 
division  managers  were  surveyed  and  asked  to  score  on  a  numerical  scale  how  they 
rated  the  relative  profit  potential  of  the  R  &  D  projects  that  were  on-going  or 
proposed.  The  scores  were  aggregated  to  arrive  at  a  number  representing  the  value 
of  a  project  relative  to  the  value  of  one  currently  being  marketed.  lis  gave  the 
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planners  a  rough  estimate  of  the  expected  profit  potential  of  the  projects  rated. 

The  scientific  department  was  then  asked  what  the  earliest,  latest  and  most  likely 
completion  times  were  for  each  project  for  various  numbers  of  scientific  teams 
assigned  to  it.  AU  of  these  factors  were  then  entered  into  a  mathematical  program 
which  resulted  in  more  efficient  manpower  utilization. 

This  example  points  out  a  significant  difference  between  modeling  industrial 
arid  military  R  &  D  programs  (or  any  budget  category,  for  that  matter).  The 
objective  function  in  industrial  models  is  generally  uni-dimensional  with  one 
motive  —  maximize  profit  in  dollars.  In  military  R  &  D  it  is  usually  meaningless 
to  assign  dollar  figures  to  the  value  of  systems.  A  model  called  MEASURE  I  which 
was  recently  developed  for  DA  by  the  Research  Analysis  Corporation  (RAC) 
follows  the  same  general  formulation  as  the  Honeywell  model,  but  suggests  a  more 
appropriate  .vay  of  quantifying  the  value  of  jysiems  [ref.  5].  Many  of  the  ideas 
that  resulted  in  the  model  presented  in  this  paper  came  from  MEASURE  I,  and 
the  author  wishes  to  credit  RAC  for  their  work.  However,  the  end  product, 
CARDBOMB,  is  a  model  quite  different  from  MEASURE  I  in  its  theoretical 


structure. 


n.  QUANTIFYING  THE  VALUE  OF  A 
RESEARCH  AND  DEVELOPMENT  SYSTEM 


Before  a  model  of  a  complex  program  can  be  formulated,  certain  parameters 
have  to  be  identified  and  measured.  This  chapter  introduces  the  concept  of  the 
value  of  a  system  undergoing  development;  what  it  is  and  how  it  might  be  quantified. 

A.  THE  CHARACTERISTICS  OF  VALUE 

Terms  such  as  value,  utility,  worth,  effectiveness  and  cost  can  be  quite  vague 
if  they  are  not  carefully  defined.  This  section  introduces  the  meaning  of  value  as 
used  in  this  paper  by  first  pointing  out  some  of  its  characteristics. 

First,  value  may  be  either  objective  or  subjective.  Objective  value  reflects 
some  genera»y  accepted  measure  like  dollars.  Subjective  value,  often  called 
utility,  can  only  indirectly  be  put  in  a  market  context.  The  value  of  an  item  becomes 
the  equivalent  of  what  a  particular  observer  is  willing  to  give  up  to  get  it.  Another 
observer  might  feel  differently  about  the  exchange. 

The  values  of  two  or  more  systems  can  also  be  commensurate  or  non-commensurate 
with  each  other.  If  the  value  of  two  items,  each  measured  on  differ  uit  scales,  can 
be  transformed  to  the  same  scale,  then  they  are  said  to  be  commensurate.  For 
objective  mea.sures  it  is  generally  a  simple  matter  to  determine  relations  between 
value  scales.  If  the  value  of  item  A  is  measured  in  dollars  and  item  B  in  cents, 
one  simply  multiplies  th.e  value  of  B  by  100  to  make  the  two  values  commensurate. 

In  the  subjective  measurement  sense  the  value  of  two  systems  can  be  considered 
commensurate  if  the  observer  can  relate  the  value  of  system  A  to  that  of  B  in  terms 
meaningful  to  him.  A  pilot  may  say  that  as  far  as  he  U  concernerl.  <>  overall 
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value  the  Army  will  get  from  the  Cheyenne  program  ii  greater  than  the  value  of 
the  Cobra.  Since  be  can  compare  these,  they  are  commensurate  to  him.  The  value 
of  two  systems  are  nomcommensurate  in  the  subjective  sense  if  the  observer  cannot 
relate  their  values.  The  same  pilot  may  have  no  feeling  for  the  value  of  the 
Cheyenne  program  relative  to  one  of  the  tank  programs  because  of  his  lack  of 
familiarity  with  the  latter.  Even  if  he  were  thoroughly  familiar  with  both,  the 
differences  in  their  missions  may  not  allow  him  to  reasonably  compare  their  values. 

A  third  characteristic  of  value  is  concerned  with  how  many  attributes  must 
be  considered  in  its  measurement.  The  value  of  a  system  is  single  attributed  if  it 
can  be  considered  to  accomplish  only  one  significant  objective,  e.g.,  generate 
profit.  It  has  multiple  attributes  if  more  than  one  significant  objective  must  be 
considered  in  its  determination.  For  example,  it  may  move,  shoot  and  communicate. 

Lastly,  nicasurcs  of  value  must  be  time  daUd  and  scenario  related.  The  value 
of  a  system  is  generally  not  considered  to  be  constant  over  time.  It  cannot  be 
expected  that  the  Cheyenne  will  have  the  sam.e  value  in  1976  as  it  will  in  1986. 
Similarly,  a  system’s  value  can  be  considerably  different  for  two  different  scenarios, 
or  operating  environments,  in  which  it  might  be  placed.  The  value  of  the  MBT  70 
tank  may  vary  considerably  depending  on  whether  it  is  performing  a  combat  mission 
in  the  Vietnam  Delta  or  the  Fulda  Gap. 

Usually  th^4-e  is  considerable  disparity  in  the  ease  by  which  value  can  be 
measured  for  industrial  projects  and  military  systems.  Risking  a  highly  oversimplified 
generalization,  it  might  be  said  that  tiie  measurements  of  value  in  business  models 
are  objective,  single  attributed  and  commensurate,  while  in  the  military  they  must 
be  subjective,  multi-attributed,  and  most  likely  only  subsets  of  systems  are 
commensurate,  i.e.,  Cheyenne  and  Cobra. 


B.  OBSTACLES  TO  MEANINGFUL  MEASURES  OF  VALUE 


If  the  value  of  a  military  R  &  D  program  is  characterired  in  this  way,  significant 
obstacles  are  raised  in  arriving  at  meaningful  measures.  CARDBOMB  uses  various 
methods  to  handle  each  of  the  obstacles  discussed  in  this  section. 

If  value  is  considered  to  be  subjective,  who  is  to  be  selected  as  the  observer  to 
render  Ids  judgement?  Strictly  theoretical  models  beg  this  issue  and  call  him  the 
decision  maker.  One  might  argue  that  this  is  the  Secretary  of  the  Army,  but  who 
can  reasonably  expect  one  man  to  have  the  necessary  knowledge  of  each  of  the 
more  than  600  projects,  or  even  the  time  to  think  about  it.  Alternatively,  some 
top  level  advisory  board  knowledgeable  of  all  tlie  systems  might  be  surveyed, 
raising  the  additional  problem  of  how  all  tlieir  replies  should  be  aggregated,  i.e., 
which  one  of  them  ha?  th«!  “right”  answer. 

The  model  builder  is  also  faced  with  probleins  caused  by  the  multiple  attribute 
nature  of  value.  Which  measures  of  effectiveness  should  be  selected?  How  are 
they  to  be  measured,  weighted  and  aggregated  to  arrive  at  a  single  value  measurement? 
What  indirect  objectives,  that  is,  extraneous  to  the  primary  mission  of  the  program 
should  be  considered?  A  glance  at  the  following  incomplete  list  will  show  that 
these  indirect  objectives  seldom  appear  in  any  cost-effectiveness  study,  but  have  a 
definite  impact  on  the  decision  problem. 

1.  Support  of  an  adequate  R  &  D  community.  If  the  budget  of  System  X 
is  substantially  reduced,  forcing  the  bankruptcy  of  Company  Y,  how  is 
the  future  R  &  D  effort  affected? 

2.  Political.  If  System  X  is  dropped,  what  will  result  from  Senator  Smith’s 
reaction? 

3.  Enemy  R  &  D  reaction.  If  we  develop  System  X,  what  can  we  expect 
the  Russians  to  do? 
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Another  obstacle  to  the  development  of  a  useful  model  of  the  RDTE  Program 
is  the  non-commensiurability  charactmistic  of  value.  To  construct  a  model,  value 
must  be  measured.  To  construct  an  optimization  model,  the  value  of  each  system 
must  be  measured  on  the  same  scale,  and  the  type  of  scale  used  is  significant.  This 
point  should  become  clearer  in  the  next  section. 

Even  within  commensurate  subsets  of  systems,  i.e.,  those  designed  to  accomplish 
generally  similar  objectives,  additional  obstacles  arise  tc  complicate  the  measurement 
of  value.  Most  systems  are  not  indeiiendent  entities  designed  to  operate  in  an 
environment  all  their  own.  More  likely  they  achieve  their  value  in  a  scenario  in 
which  they  operate  with  other  systems.  Therefore  the  values  of  two  or  more  systems 
may  be  interdependent  rather  than  .independent.  The  value  of  Tank  System  A 
might  be  considerably  different  when  Tank  System  B  is  developed  and  when  it  is  not. 

Still  more  questions  must  be  addressed  in  considering  how  to  measure  value. 
How  should  the  three  types  of  uncertainty  be  considered  in  the  measurement  of 
value?  Should  the  value  of  a  system  be  measured  in  the  worst  scenario  envisioned, 
or  the  most  likely?  Should  a  conservative  or  optimistic  view  be  taken  in  predicting 
the  operating  characteristics  of  the  resulting  system?  And  similarly,  how  should 
cost  uncertainty  be  considered,  since,  as  has  been  pointed  out,  an  ir.correct  estimate 
may  effect  a  loss  in  value  to  the  program  or  other  programs? 

This  list  of  obstacles  illustrates  the  magnitude  of  the  problem  of  measuring 
value.  All  of  these  considerations  must  be  addressed  in  determining  how  well  this 
or  any  other  R  &  D  model  represents  the  real  world.  Following  the  development 
of  the  value  measurement  segment  of  thb  paper,  we  will  return  to  this  list  to 
discuss  how  each  obstacle  was  handled. 
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C.  A  TAXONOMY  OF  RESEARCH  AND  DEVELOPMENT  SYSTEMS 


In  moving  toward  the  goal  of  finding  a  method  for  making  the  value  of  as 
many  ROTE  projects  commensurate  as  possible,  a  taxonomy,  or  classification 
scheme,  must  first  be  defmed  whereby  “  reasonably  commensurate  siibsets”  of 
systems  can  be  identified.  A  reasonably  commensurate  subset  will  be  loosely 
defined  as  a  group  of  systems  aimed  toward  the  accomplishment  of  generally 
similar  missions  or  objectives. 

The  most  logical  place  to  begin  such  a  taxonomy  is  with  the  one  currently 
used  by  the  Army  and  defined  in  the  Army  Strategic  Objectives  Plan  (ASOP).  It 
identifies  each  RDTE  line  item  project  as  belonging  to  one  of  the  following  24 
categories. 

1.  Air  Mobility 

2  Air  Defense 

3.  Tank/Antitank 

4.  Communications 

6.  Surveillance,  Target  Acquisition,  and  Night  Operations  (STAND) 

6.  Surface  Mobility 

7.  Indirect  Fire 

8.  Infantry  Weapons 

9.  Logistic  Support 

10.  Electronic  Warfare 

11.  Command  and  Control 

12.  Chemical/Biological 

13.  Nuclear 

14.  Ballistic  Missile  Defense 

16.  Personnel,  Care,  Protection  and  Survival 

16.  Human  Performance 

17.  Counterinsurgency  and  Special  Warfare 

18.  Environmental  Analysis  of  Military  Operations 

19.  Mapping 

20.  Construction  Methods 

21.  Research  for  More  than  One  of  the  Above 

22.  Research  Associated  with  None  of  tlie  Above 

23.  Testing 

24.  Management  and  Other  Support 

Categories  1  through  14  are  called  Hardware  Categories,  15  Uiiough  10  are  Non- 
Hardware  Categories,  and  21  through  24,  Support  Categories,  i'his  paper  will  use 
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the  same  classification  scheme  with  the  following  modifications.  Eliminate  category 
21  and  assign  its  projects  to  the  most  valuable^  system  it  supports.  Remove 
category  23  and  assign  its  projects  to  the  category  of  the  system  it  is  designed  to 
test.  Eliminate  category  22  and  assign  its  projects  to  a  new  category  called  Basic 
Research.  This  category  will  have,  in  addition  to  those  projects  from  the  old  category 
22,  all  basic  research  projects  which  are  not  a  part  of  some  development  plan. 

For  example,  there  may  be  a  basic  research  project  in  support  of  the  development 
of  M3T  70.  This  project  would  be  assigned  to  category  3,  Tank/Antitank.  Another 
basic  research  project  may  be  aimed  toward  category  3  but  not  part  of  some 
development  plan.  It  would  be  assigned  to  the  new  category,  Basic  Resea^’ch. 

Most  projects  that  are  called  basic  research  or  exploratory  development  would  fall 
into  this  new  Category  22.  To  summarize,  the  categories  for  this  paper  are: 

Categoric:  1  through  20;  no  change. 

Categories  21  through  24;  eliminated  and  replaced  by, 

21  Management  and  Other  Support 

22  Basic  Research 

Now  that  a  taxonomy  has  been  defined,  all  that  remains  is  to  decide  what 
belongs  in  each  category.  As  was  previously  mentioned,  the  Army  currently 
identifies  each  line  item  project  as  belonging  to  one  of  the  categories.  What  is 
required  for  this  model,  however,  is  that  systems  be  assigned  rather  than  projects. 

Although  this  idea  has  been  previously  introduced,  this  last  sentence  requires 
considerable  expansion.  Up  to  this  point  the  terms  "project”  and  “system”  have 
been  used  more  or  less  interchangeably.  In  most  of  the  Systems  Analysis  literature 
on  capital  budgeting,  a  project  refers  to  that  entity  which  results  in  value.  In  the 
RDTE  budget,  a  project  is  usually  one  of  a  group  of  contracts,  all  of  which  go 

“Most  valuable  will  be  explicitly  defined  later. 
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into  the  development  program  which  hopefully  results  in  an  operational  system. 

In  this  model  it  is  the  value  of  the  system  which  we  seek  to  measure,  not  the 
projects.  To  avoid  this  ambiguity  a  system  will  be  defined  as  one  or  more  RDTE 
line  item  projects  or  portions  of  projects  which,  when  taken  together,  expect  to 
result  in  value  to  the  Army.  Thus  the  Cheyenne  system  from  category  1  might 
be  defined  as  consisting  of  three  projects;  advanced  development,  test  and  evaluation, 
and  operational  systems  development.^  This  discussion  pertaining  to  the  assignment 
of  systems  to  categories  applies  only  to  categories  1  through  20.  Categories  21 
and  22  are  treated  differently  than  the  others  in  the  model. 

One  additional  classification  will  be  assigned  to  each  system.  The  system  will 
be  further  identified  as  belonging  to  one  of  three  time  periods  depending  on  when, 
under  reasonable  funding  levels,  it  can  be  expected  to  reach  its  operational  readi¬ 
ness  date  (or,  for  Non-Hardware  Categories,  the  oojectives  of  the  res,carch  will  be 
reached).^®  The  time  periods  are  arbitrarily  defined  as: 

T]^  :  The  five  year  period  commencing  with  the  fiscal  year  for  which  the 

budget  is  being  formulated  (hereafter  called  1973). 

T2  :  The  five  years  following  Tj 

T3  :  The  five  year  period  following  the  end  of  T2. 

To  summarize  this  classification  scheme,  with  the  exception  of  projects 
assigned  to  Categories  21  and  22,  all  other  projects  will  be  identified  wi<h  a  single 
system.  Each  system  will  in  turn  be  assigned  to  a  category  and  a  time  period. 


®The  task  of  defining  systems  should  be  made  easier  by  referring  to  the 
appropriate  Development  Concept  Paper  when  one  exists. 

precise  definition  of  how  a  system  is  time  period  classifii  deferred 
until  Chapter  3. 
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D.  MEASURING  THE  VALUE  OF  SYSTEMS  WITHIN  A  CATEGORY 


Having  discussed  tlie  characteristics  of  value,  we  now  turn  to  defining  and 
measuring  it.  The  value  of  an  R  &  D  system  will  be  loosely  defined  as  a  notional 
(subjective)  judgement  made  by  an  individual  when  he  considers  the  capabilities 
of  the  system  for  meeting  a  specific  need  (or  set  of  needs).  Since  this  judgement  is 
made  in  relation  to  the  capabilities  of  other  systems,  it  is  often  called  relative 
value. 

Many  methods  have  been  devised  for  measuring  these  subjective  value  judge¬ 
ments.^^  In  general  they  vary  widely  in  the  ease  with  which  they  can  be  applied. 

One  of  the  easiest  to  apply  in  terms  of  the  time  required  is  a  “ranking-rating” 
method.  Briefly,  a  survey  respondent  is  first  asked  to  ordinally  rank  the  systems 
in  order  of  importance.  After  this  has  been  accomplished  he  is  asked  to  assign  a 
number  beiween  zero  ard  ten  to  the  value  of  eacn  system,  where  ten  represents 
the  value  of  the  system  deemed  most  important.  The  assumption  is  then  made 
that  these  numbers  represent  an  interval  scale  of  measurement  of  the  relative  values 
of  the  systems.^^  Due  to  the  relative  simplicity  of  this  method,  it  will  be  the 
one  described  in  this  paper.  However  the  model  can  be  used  with  value  parameters 
obtained  from  other  methods,  provided  that  they  can  be  considered  to  be  from  an 
interval  scale  of  measurement. 

Assume  for  the  moment  that,  contrary  to  reality,  the  value  of  each  of  the 
systems  within  a  category^^  and  time  period  has  a  single  attribute  and  the  attribute 

^^Burington  (ref.  6)  discusses  the  meriU  and  shortcomings  of  some  of  the 
methods  that  have  been  applied  in  the  R  &  D  conte.xt.  Fishburn  [ref.  7]  reviews  24 
different  measurement  methods  that  have  appeared  in  the  OR  and  Economic  literature. 

^^he  implications  of  this  assumption  are  discussed  in  Ap;)endi.x  A. 

^^he  discussion  of  the  value  of  systems  within  a  category  wil  -xclude  Category 
21,  Managoinenl  and  Support,  and  Category  22,  Biisic  Research.  1 1  e  are  treated 
separately  in  Chapter  4. 
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is  definable  in  some  unit  of  measure,  W.  It  could  then  be  said  that  the  i^^  system  in 

the  j***  category,  a  time  period  T  system,  has  value  during  this  period  T  of  Vy  ®  y  W, 

where  y  is  some  non>negative  real  number.  If  this  were  an  industrial  model,  W 

might  be  a  dollar  unit  of  measure  and  y  the  number  of  dollars,  resulting  in  a 
T 

value  such  as  Vy  =  S26  dollars.  The  fact  that  such  a  real  numbered  value  like 
326  dollars  cannot  be  measured  for  military  R  &  D  systems  will  be  shown  to  be 
immaterial. 

Suppose  that  a  group  of  high-level  DA  planners,  all  experts  in  the  systems  of 
the  category,  were  surveyed  and  asked  to  ordinally  rank  the  value  of  the  time 
period  T  systems  of  the  category  considering  the  following  factors: 

1.  The  performance  characteristics  of  the  resulting  system  will  be  exactly 
as  specified  in  its  current  DCP. 

2.  The  system’s  development  costs  are  immaterial. 

3.  The  system’s  operational  readiness  date  will  be  as  specified  in  the  DCP. 

4.  The  system’s  procurement  and  operating  costs  are  a  significant  consideration. 

6.  The  threats  these  systems  may  have  to  face  are  also  a  significant 
consideration. 

Suppose  further  that,  once  an  ordinal  ranking  was  established,  this  same  group 
of  experts  could  respond  to  a  second  survey  asking  them  to  rate  the  value  of  each 
system  on  a  scale  of  zero  to  ten,  where  the  value  of  the  moot  important  system  is 
taken  to  be  ten.  Assuming  the  response  to  this  survey  establishes  an  interval  scale 
of  value,  we  then  have  a  measure  of  the  value  of  the  i^^  system  relative  to  the  value 
of  the  system,  where  the  system  is  the  one  deemed  most  important. 

Arbitrarily  define  this  most  important  q^^  system  as  the  time  period  numeraire. 

l^The  numeraire  system  is  that  system  to  which  all  the  others  ,re  measured 
relative  to;  the  denominator  of  tlie  ratios. 
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We  then  have  for  each  of  the  20  categories  and  S  time  periods,  a  set  ol'  real  numbers 


between  zero  and  one; 


V^. 

V^. 

qj 


for 


i  "  1»  •  •  •  * 

T  -  1,  2,  3 
j  ■  1  , .  .  .  ,  20 


where  nj  ^  >  the  number  of  time  period  T  systems  in  the  category.  For  example, 
for 

j  ■  3  Tank/Antitank 

T  =  1  Systems  which  are  first  available  during  the  time  frame  73-77. 
i  >  2  Arbitrary  assignment.  Suppose  it  is  “Antitank  Gun  Killer”, 
q  “  1  MBT  70  (Assuming  it  was  deemed  most  important) 


Then 


^2.3  ^ 


'1,3 


0.9 


would  imply  that  the  consensus  of  the  subjective  value  judgement  of  the  group  of 
Tank/ Antitank  experts  determined  that  the  value  of  having  Antitank  Gun  Killer 
during  the  time  frame  1973-77  was  only  90  percent  of  the  value  of  having  the 
MBT  70  tank.^^ 


Once  these  parameters  are  established  it  is  then  desirable  to  get  n  measure  ol 

how  the  value  of  each  system  is  degraded  in  time  periods  subsequent  to  the  system’s 

introduction  into  the  inventory,  considering  the  most  likely  technological  obsoles- 

ence  and  enemy  counter  R  &  D  moves.  For  example,  the  Tank/Antitank  experts 

2  3 

might  respond  with  time  degradation  factors  of  aj  3  *  0.66  and  a  j  3  =  0.1, 
implying  that  the  value  the  Army  will  achieve  from  MBT  70  during  the  time 
frame  1978-82  is  only  two-thirds  what  it  was  in  1973-77,  and  during  1983-87, 

only  one-tenth  of  what  it  was  in  1973-77. 

^^How  J  consensus  measure  is  reached  discussed  later. 
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OiA  further  set  of  data  is  required  to  make  the  value  measurements  within  the 


category  commensurate.  The  systems  within  a  category  were  partitioned  into 
time  periods  to  ease  the  problem  of  the  survey  respondents  requirement  to  relate 
systems’  values  being  developed  for  different  time  frames.  However  they  now  must 
be  siuveyed  to  relate  the  values  of  the  most  important  systems  from  each  period, 
the  time  period  numeraires.  Let  the  time  period  1  numeraire  be  the  denominator 
in  these  two  measurements  (now  called  the  category  numeraire)  and  measure  the 
values  of  T2  and  T3  numeraires  relative  to  its  value  in  those  time  periods.  That 
is  measure; 


a^  -V  ^  ■ 
®qj  qj 


and 


qj 


a^  V  ^  • 

qj  qj 


for  i  =  1,  .  .  ,  ,  20 


The  simple  formula  for  relating  the  values  of  all  systems  within  the  j  category 
relative  to  its  category  numeraire  is  then, 


VT. 

qj 

Jwy 

qj  qj 


for 


i  =  1,  .  .  .  ,  nj -p 

T  =  1,  2,  3 
j  =  1 . 20 

a^  ■  =  1 

®qj  ^ 


(1) 


Before  continuing  the  development  of  the  model  it  may  be  beneficial  at  this 
point  to  consider  how  such  data  might  be  obtained.  One  of  the  most  widely 
researched  techniques  is  the  Delphi  Method  developed  by  Dalkcy  of  the  Rand 
Corporation  [ref.  8].  It  could  be  applied  to  this  model  in  the  following  way. 

Write  a  carefully  worded  survey  that  included  a  precise  list  of  the  factors  the 
respondents  were  to  consider.  Part  of  this  list  would  be  the  performance  character¬ 
istics  and  other  information  from  the  DCP  of  each  of  the  systems  in  this  category. 
Select  at  lea.st  ten  respondents  considered  to  be  exjjerts  in  the  syst>  s  of  this 
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category  and  issue  them  the  survey  individually.^^  Gather  their  responses  and 
record  how  each  of  them  rated  the  relative  values  and  time  degradation  factors. 

Reissue  the  survey,  but  this  time  inclose  the  distribution  of  responses  given  for 
the  last  survey,  and  ask  the  respondents  to  reconsider  their  initial  reply.  Also 
ask  that  if  they  continue  to  rate  a  parameter  outside  the  interquartile  range  of 
the  group  distribution  of  responses,  to  add  a  brief  comment  why  they  did  so.^*^ 

Reissue  the  survey  again,  inclosing  the  new  distribution  of  responses  and  all  the 
anonymous  comments,  and  ask  for  a  final  consideration.  Dalkey  has  shown  in 
a  number  of  tests  of  the  Delphi  Method  that  considerable  convergence  in  responses 
occurs  after  as  few  as  three  iterations  of  this  controlled  feedback  technique. 
Unfortunately  there  is  no  way  of  knowing  whether  or  not  it  is  converging  to  the 
“true”  parameter,  some  measure  of  the  relation  of  the  intrinsic  values  of  the 
systems. 

Since  the  model  will  require  a  single  measure  to  represent  what  is  essentially  a  random 
variable  parameter,  this  paper  will  consider  the  mean  of  the  distribution  of  responses  to  be 
that  measure,  since  in  a  statistical  sense  it  represents  the  average  of  the  responses.  However, 
for  the  sensitivity  analysis  on  the  parameters  discussed  in  Chapter  4,  the  end  point  of  the 
interquartile  range  should  also  be  recorded  to  provide  the  user  with  a  feeling  for  the  var¬ 
iance  of  opinion  of  his  expert  advisors. 

E.  COMMENSURABILITY  BETWEEN  CATEGORIES 

The  data  collection  effort  outlined  in  the  last  section  resulted  in  commensurate 
measures  of  value  within  each  category,  i.e.,  the  value  of  the  systems  within  the 
category  are  measured  on  the  interval  scale  of  value  of  the  category  j  numeraire 

1  fi 

*°In  [ref.  8)  Dalkey  shows  that  the  individual  survey  is  preferred  to  a  group 
meeting  survey  in  that  it  eliminates  a  possible  bias  caused  by  one  or  more  dominant 
personalities. 

1 7 

The  interquartile  rujige  is  that  interval  containing  the  middle  50  percent  of 
the  responses 
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This  method  requires  no  mathematical  assumptions,  but  presumes  that  each  of  the 


survey  respondents  are  thoroughly  familiar  with  the  development  programs  of  each 
of  the  category  numeraire  systems. 

The  secondary  method  requires  the  assumption  that  the  value  parameters 
determined  for  the  systems  within  a  category  be  additive,  i.e.. 


V|J  *  ''tj 
'^qj  ''qj 


This  assumption  means,  for  example,  that  if  the  values  of  systems  i  and  K  were 

rated  at  '/6  the  value  of  the  category  numeraire,  then  the  value  of  having  both  the 

1  d 

i  and  k  systems  is  the  same  as  the  value  of  having  the  numeraire  system. 

With  the  assumption  of  additivity  of  value  the  following  simple  equation  can 
be  formulated. 


y  V-  •  •  vf  =  N- 

j  J 

1=1 


for  j  =  1,  .  .  .  ,  20 


where  Vj  represents  the  “slack  vaiue”  for  category  j,  and  Nj  is  a  measure  of 
aggregated  needs.  In  words  Equation  3  says  that  the  needs  in  this  category  equal 
the  sum  of  the  values  of  all  the  systems  being  developed,  minus  some  slack  value. 
While  it  is  recognized  that  “needs”  are  really  multi-dimensional,  (i.e.,  in  category 
1  the  Army  has  a  need  for  both  an  attack  helicopter  and  a  .surveillance  helicopter) 
it  is  felt  that  the  concept  of  aggregated  needs  is  not  an  excessive  abstraction  since 


in  the  ASOP  it  is  currently  assigned  a  priority.  Vj  is  included  to  cover  two 
possibilities.  First,  due  to  technological  uncertainty  in  the  systems  being 
developed,  it  is  cxinceiveablc  that  as  a  hedge  against  this  uncertainty  more 
systems  are  being  developed  than  are  e:;pected  to  be  required.  In  this  case 

^^Further  implications  of  thi.s  u.ssumption,  as  well  a'  method.s  r  testing  its 
validity,  are  discussed  in  .Appendix  .A. 


29 


Vj  would  be  a  positive  quantity.  Alternatively,  intelligence  on  a  recent  R  &  D 
breakthrough  by  a  potential  enenv  may  have  suddenly  increased  Nj,  and  sufficient 
time  has  not  yet  passed  to  reach  the  concept  formulation  stage  for  one  or  more 
new  systems.  Here  Vj*  would  be  a  negative  quantity.  Obtaining  a  measure  of  V* 
might  be  accomplished  in  the  following  way.  Again  using  the  Delphi  technique, 
survey  the  category  experts  with  questions  similar  to,  “Assuming  the  development 
of  all  systems  proceeds  as  in  their  DCP’s,  what  system  could  be  dropped  from 
the  program?”  If  the  consensus  were  the  system,  it  would  imply  that 

Vqj  • 

Alternatively,  the  consensus  might  be  that  no  system  can  be  dropped,  implying 
that  either  the  needs  are  exactly  met  or  a  gap  between  value  and  needs  exists. 

The  value  oi  tnis  negative  slack  might  be  deternmicd  by  asking,  “If  in  your  opinion 
a  system  has  to  be  added  to  the  program  to  meet  the  needs  in  this  category,  to 
the  value  of  which  system  currently  in  the  program  should  the  value  of  this  new 
system  be  equal?”  If  the  consensus  were  the  1^^  system,  then 

''qj  ''qj  ' 

A  very  useful  equation  can  result  from  taking  the  ratio  of  Equation  3  for  two 
different  categories,  in  particular  category  1  and  category  j,  and  applying  some 
algebraic  manipulations. 


I  Vy  .  v;  Nj 


Multiplying  both  sides  of  this  equation  by  the  quantity 


yields 


Nl 


qj 

^q.l 


^qJ 

'^q.l 


Ni 


3L 

^q.l 


Z  ■  vj 
1 _ 


(4b) 


and  since 


Vqj 


1/y 


q,i 


i/v. 


q.i  ‘'’qj 

substituting  for  the  second  term  on  the  right  in  Equation  4b  we  get 


^q.i 


Ni 


z  ^ 


‘  '^q.l  '"q.l 


Z  ^ 


qj 


qo 


(4) 


Note  that  all  the  terms  in  the  bracket  in  Equation  4  have  already  been 
determined.  They  are  the  values  of  the  systems  (oi  slack  value)  relative  to  their 
category  numeriares.  The  term  on  the  left,  the  value  of  the  category  j  numeraire 
relative  to  tire  value  of  the  program  numeraire,  is  that  quantity  required  for  the  use 
of  equation  2.  The  term 


N, 


howev  ?r,  is  unknown.  This  represents  the  need  during  time  period  1  for  new 
category  j  systems  relative  to  the  need  for  Air  Mobility  systems.  Determining 
these  19  pjarameters  might  also  be  done  using  the  Delphi  technique  ith  a  group 
of  high  level  DA  managers.  As  waj  mentioned,  it  is  currently  done  .a  the  ASOP 
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on  an  ordinal  scale  for  the  first  14  Hardware  Categories.  The  extension  that  this 
secondary  approach  would  require  would  be  to  ordinally  rank  all  20  categories, 
and  then  determine 


in  a  manner  similar  to  that  used  to  find 


VqJ 


F.  AN  ALTERNATE  APPROACH  TO  MEASURING  VALUE 

In  developing  a  useful,  working  model,  trade-offs  must  be  made  between  how 
much  information  the  model  can  provide  the  user  in  his  decision  problem  and  how 
much  effort  is  required  in  data  collection.  The  approach  outlined  thus  far  already 
requires  a  considerable  data  collection  effort,  and  even  more  is  asked  for  in  the 
next  chapter.  It  should  be  pointed  out  that  the  parameters  required  to  be  obtained 
by  survey,  arc  for  the  most  part,  the  same  as  those  needed  for  RAC’s  model, 
MEASURE  I.  This  was  purposely  done  to  minimize  the  difference  in  the  data 
collection  effort.  It  should  also  be  noted,  however,  that  how  the  data  are  used 
to  arrive  at  commensurate  measures  of  relative  value, 


is  considerably  different  for  tire  two  models.  This  point  is  discussed  further  in  the 
next  section. 

Chapter  5  of  this  paper  outlines  a  manage.ment  information  system  that  might 
be  used  to  implement  CARDBOMB.  Mention  is  made  that  the  user  as  the 
option  to  override  any  of  the  value  parameters  obtained  thus  far  and  insert  his 


32 


own  relative  values;  in  essence,  disagreeing  with  his  advisors.  However,  the  approach 
outlined  in  the  last  two  sections  gives  him  little  insight  into  why  the  survey  respondents 
replied  as  they  did.  The  following  approach  might  provide  more  insight,  but  at 
the  cost  of  significantly  increasing  the  data  coUection  effort. 

Recall  that  the  value  of  a  system  and  the  need  for  it  are  both  multi-attributed. 
The  primary  approach  requires  a  survey  respondent  to  consider  a  vector  of  perfor¬ 
mance  characteristics  as  well  as  a  vector  of  possible  scenarios,  weight  the  impoilance 
of  each  characteristic  and  the  likelihood  of  each  scenario,  and  aggregate  all  these 
factors  in  his  head  to  arrive  at  a  single  measure  of  relative  value.  Hence  a  great 
deal  of  information  is  not  available  to  a  user  of  the  model  should  he  wish  to  con¬ 
sider  altering  the  value  parameters  of  one  or  more  of  the  systems.  Some  of  this 
Insight  might  be  regained  by  using  the  following  alternate  approach.  Although 
it  is  outlined  only  for  obtaining  the  relative  values  of  systems  within  a  category 
and  time  period,  it  could  also  apply  to  obtaining 


(depending  on  which  method,  the  primary  or  secondary,  was  used). 

Survey  the  category  j  experts  to  obtain  a  list  of  what  they  consider  the 
significant  performance  characteristics  of  the  systems  in  the  category  to  be. 
Label  these  k  =  1,  .  .  .  ,  m.  Follow  this  with  a  survey  to  determine  the  relative 
importance  of  each  of  these  characteristics,  relative,  for  example  to  k  “  1.  Label 
these  weighting  factors  a.s  bj^,  where  bj  -  1.  Again  survey  the  category'  experts  to 
determine  the  value  of  each  of  the  systems  relative  to  the  viduc  of  the  category 
numeraire  for  each  of  the  characteristics.  Label  these 
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With  the  additivity  assumption  it  could  then  by  hypothesized  that  the  value  of  the 
i^^  system  was  equal  to  the  sum  of  its  value  in  achieving  each  performance 
characteristic,  weighted  by  the  importance  of  the  characteristic;  i.e., 


^  k-1 


We  would  then  have: 


''ij 

“  ''i,k 

Vqj 

’  1 

k=l  1'k 

“  ’'i.k 

"  ''q.k 

m 

k=i 


for  i  =  1, ,  .  ,  ,  n;  ff 
i-l . 


A  simple  example  of  this  approach  might  help  to  clarify  its  use.  Suppose  the 

three  surveys  mentioned  above  for  category  1,  Air  Mobility,  resulted  in  the  following 

set  of  data.  Survey  1  determined  that  four  important  characteristics  should  be 

considered  for  Air  Mobility  systems. 

k  =  1  Air  speed 

k  “  2  Maneuverability 

k  *  3  Weapons  accuracy 

k  "  4  PEMA  and  OMA  costs 

Survey  2  determined  that,  considering  the  future  needs  for  air  mobility  systems, 
the  importance  of  each  of  these  characteristics  relative  to  air  speed  (determined 
to  be  most  important)  were 

bf  “1,  b2  ■  0.9,  bg  -=  0.8,  b,j  “  0.7 
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Survey  3  resulted  in  ttie  determination  that  the  value  of  the  design  specifications  and 
costs  for  system  2  relative  to  the  value  of  those  for  system  1,  the  category  numeraire 
were, 


''1.1 


0.6, 


1.3, 


implying,  for  example  that  considering  air  speed  alone,  system  1  is  twice  as  valuable 


as  system  2.  Using  Equation  5  to  aggregate  this  data  we  would  have. 


^2,1  (1)  (0.5)  +  (0.t>)  (1.2)  +  (0.8)  (0.8)  +  (0.7)  (1.3)  _  .. 

- - - -  0.9  (to  the 

Vl,l  1  *  0.9  t  0.f  .  0.7  tenth) 


Because  of  the  additional  data  collection  effort  that  would  be  required,  this 
approach  is  considered  by  the  author  to  be  inappropriate  for  this  model.  Discussion 
of  it  has  been  included  in  this  paper  primarily  to  indicate  a  limitation  to  the 
approach  discussed  in  the  previou.s  two  sections  the  limitation  of  the  lack  of 
insight  provided  the  user  in  understanding  the  factors  that  were  considered  in 
aniving  at  the  relative  value  and  relative  category  needs  parameters.  This  limitation 
might  be  partly  overcome  by  having  the  survey  respondents  submit  a  brief 
paragraph  on  why  they  decided  on  the  responses  they  gave.  These  might  be 
summarized  in  a  short  paper  for  each  system  and  referred  to  by  the  user  when  he 
is  considering  whether  or  not  he  agrees  with  a  particular  parameter. 


G.  COMMENTS  ON  THE  MEASUREMENT  OF  VALUE 

Section  B  of  this  chapter  outlined  obstacles  to  the  effective  measurement  of 
value  for  R  &  D  systems.  It  should  be  apparent  that  the  value  measurement 
portion  of  this  model  has  dealt  with  these  problems  in  various  ways.  In  this 
section  each  of  these  obstacles  is  recalled  in  order  to  provide  the  re;  !er  with  more 
information  on  wliich  to  judge  ttie  validity  of  the  methods  used. 
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Lack  of  a  unit  of  measure.  Recall  that  it  was  assumed  that  a  unit  of  measure, 
W,  existed,  and  that  the  value  of  each  system  could  be  measui'ed  as  having  some 
real  number,  y,  of  these  units.  On  an  interval  scale  of  measure,  W  cancels  out  of 
the  equation 

Vqj’  S',"  ’ 

and  the  result  is  some  dimensionless  real  number  like  0.8.  When  a  program 
numeraire  (the  denominator  of  all  the  ratios)  is  selected,  it  can  then  be  said 
that  the  value  of  all  the  systems  are  measured  on  the  value  scale  of  this 
numeraire.  For  example 
V: 


^q.l 


=  0.8 


is  equivalent  to  V;  j  =  0.8  •> ,  implying  that  whatever  real  number  is  selected  to 

represent  Vg  j,  Vy  must  always  be  0.8  of  that  number. 

The  subjective  nature  of  value.  In  any  analytical  study  there  is  a  natural  aversion 
to  using  subjective  value  judgements  as  important  parameters.  Nonetheless,  very 
few  decision  problems  exist  where  they  are  not  required  to  be  made.  Regardless 
of  the  pains  to  which  a  decision  maker  may  go  to  collect  “objective  facts”  bearing 
on  the  problem,  he  ultimately  must  subjectively  weigh  these  facts  when  faced  with 
a  tradeoff  decision.  This  model  assists  him  by  presenting  him  with  a  consensus 
measure  of  the  subjective  value  judgements  of  his  advisors,  but  allowing  him  to 
override  their  advice  and  use  his  own  judgement*.. 

The  aggregation  of  subjective  value  judgements.  Regardless  of  tiic  amount  of 
objective  information  available  to  a  group  of  experts,  when  they  apply  their  own 
judgements  it  is  unlikely  that  they  will  exactly  agree.  Tlie  Delphi  M  ;hod  for 
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aggregating  their  advice  is  used  in  this  model  because  it  represents  the  extent  of 


the  state  of  the  art  in  what  might  be  called  “opinion  technology/’  To  keep  the 
model  reasonably  simple,  a  single  measure  of  value,  the  mean  of  their  responses, 
is  used.  However  the  decision  maker  has  the  right,  indeed  the  need,  to  know  the 
extent  of  the  disagreement  of  his  advisors.  Hence  the  end  points  of  the  interquartile 
range  of  their  distribution  of  responses  will  be  recorded  as  a  measure  of  the  variance 
of  their  opinion. 

The  non-commensurable  nature  of  value.  The  model  provides  for  partitioning 
the  systems  of  the  RDTE  program,  first  into  categories  of  systems  which  perform 
similar  missions  and  then  into  time  periods.  After  value  judgements  are  made  on 
these  reasonably  commensurate  subsets,  mathematically  consistent  methods  are 
applied  to  relate  the  value  of  each  system  to  the  value  scale  of  the  program 
numeraire.^®  It  was  .statod  earlier  that  these  niatliematical  methods  are  considerably 
different  in  this  model  and  MEASURE  I.  RAC’s  model  provides  for  the  weighting 
of  the  relative  value  parameter  within  the  j^^  category  by  multiplying  it  by  the 
relative  need  parameter  of  that  category.  In  the  notation  of  this  paper  this  can  be 
represented  by 
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This  produces  a  different  result  than  that  obtained  by  using  Equations  2  and  4,  as 
the  following  simple  example  demonstrates. 

Assume  a  two  category  program  with  two  systems  per  category  where  there 
exists  absolute  measures  of  value  and  need  that  are  known  with  certainty  as  follows. 


^^Besides  identifying  reasonably  commensurate  subsets,  the  p;\  ioning  helps 
to  eliminate  bi.-is.  A  proponent  of  Air  Mobility  does  not  rate  the  (  enne 
relative  to  Ml’.T  70. 


3? 


Category  1 


20 


Category  2 


^^2.1 

Nl 

Vl.2 

^^2,2 

N2 


=  10 

=  26,  implying  vj  =  20  +  10  -  25  =  5 
=  15 


»  10 

=  20,  implying  V2  =  15  +  10  -  20  =  5 


Although  it  is  apparent  that 


10 

20 


let  us  attempt  to  verify  this  using  the  methods  of  both  models.  Equation  4 
results  in; 


20  10  5 

20 

20  ^  20  '  20 

25 

16  10  5 

15  15*15 

15 

20 


and  using  this  in  Equation  2  yields; 

^22  10  15  10  . 

- =  —  •  —  =  —  ,  m  agreement  with  me  data. 

15  20  20 


The  MEASURE  1  method  would  result  in, 

V22  V22  N2  10  20  8 

-  *  -  •  —  =  —  •  —  =  —  ,  not  in  agreement  with  the  data. 

Vii  V12  15  25  15 

This  inconsistency  is  not  the  result  of  incorporating  the  concept  of  slack  value. 

Even  assuming  Vj  and  are  zero,  implying  =  30  and  N2  =  25,  the  RAC 


method  results  in 
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^  ,  19. .  .  i  19. 

Vjl  ’  15  30  '  9  ^  20 

The  multi-attributed  nature  of  value.  This  point  was  discussed  at  length  in 
Section  F  where  an  alternate  approach  to  measuring  value  was  outlined.  The 
principal  approach  requires  the  survey  respondent  to  consider  multiple  attributes, 
but  reduce  them  to  a  single  measure  of  relative  value.  As  mentioned,  significant 
information  may  be  lost  to  the  user  of  the  model  when  he  desires  to  question  the 
validity  of  a  value  parameter. 

Value  as  a  function  of  time.  The  model  requires  that  the  category  experts 
estimate  value  degradation  parameters  reflecting  the  percent  of  the  initial  value 
remaining  in  time  periods  subsequent  to  the  time  period  when  the  system  is 
expected  to  be  operational.  Although  five  year  time  periods  were  defined,  this 
is  completely  arbitrary. 

Scenario  risk.  This  becomes  a  factor  for  consideration  by  the  survey  respondents, 
and  thus  is  not  made  visible  in  the  model.  The  respondent  must  consider  how  the 
systems  would  perform  in  various  circumstances,  and  then  apply  his  own  judgement 
concerning  the  likelihood  of  these  scenarios.  Thus,  if  all  other  factors  were  equal ^ 
if  he  considered  a  counterinsurgency  war  more  likely  than  a  major  NATO  encounter, 
the  system  designed  for  the  former  would  have  more  value  to  him  than  one  designed 
for  the  latter. 

Cost  unce.”tainty.  Except  as  this  factor  might  be  considered  by  a  survey 
respondent,  this  important  consideration  is  not  included  in  the  model.  This 
applies  also  to  indirect  objectives,  e.g.,  effects  on  the  R  &  D  community,  political, 

^^This  discus..ion  is  not  to  imply  that  this  is  the  only  differenct  in  the  value 
determination  methods  of  the  two  models. 
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etc.  Chapter  5  does  outline,  however,  how  sensitivity  analysis  con  be  conducted  on 
cost  parameters. 

Technological  uncertainty.  Recall  that  the  value  determination  surveys 
directed  the  respondent  to  assume  that  performance  characteristics  and  operational 
readiness  dates  would  be  exactly  as  projected  in  tlie  current  DCP’s.  Thus  value 
was  measured  independent  of  technological  uncertainties.  The  next  chapter 
introduces  how  the  technological  factors  are  considered. 

The  interdependence  of  systems’  values.  It  would  be  unreasonable  to  require 
the  Air  Mobility  experts  to  envision  a  future  where  the  Army  operated  with  only 
one  system  from  this  category.  Therefore  the  value  parameters  have  a  certain 
degree  of  interdependence.  Yet  the  mathematical  formulation  of  the  model  will 
require  that  these  values  be  assumed  independent.  This  inconsistency  has  important 
implications  concerning  the  solutions  the  model  generates,  particularly  with  regard 
to  the  usefulness  of  the  model  in  providing  insight  into  which  systems  might  be 
dropped  from  the  program.  Discussion  of  these  implications  is  deferred  until 


Chapter  4. 


ni.  RELATING  VALUE  TO  COST 


The  development  of  the  model  has  thus  far  dealt  solely  with  methods  for 
determining  commensurate  measures  of  value  for  the  RDTE  systems.  The  reader 
may  have  already  formed  an  opinion  concerning  the  usefulness  and  validity  of 
those  methods.  Should  the  concepts  developed  in  the  prev  ious  chapter  be  rejected, 
the  remainder  of  the  model  cari  still  be  useful,  providing  that;  (1)  some  method 
is  used  to  arrive  at  commensurate  measures  of  value  of  the  systems  as  currently 
planned,  and  (2)  some  measure  of  how  that  value  changes  over  time  is  determined. 

Recall  that  in  the  previous  chapter,  survey  respondents  were  directed  to 
disregard  RDTE  costs  and  technological  uncertainty  in  determining  the  value 
parameters.  We  now  turn  to  a  consideration  of  these  two  factors  in  order  to 
arrive  at  an  intermediate  goal  —  determining  how  the  value  a  system  is  expected 
to  achieve  varies  with  RDTE  dollar  costs. 

A.  SOME  CONSIDERATIONS  ON  COST 

Costs  are  incurred  in  the  implementation  of  the  RDTE  program  for  a  number 
of  resources,  e.g.,  manpower,  materiel,  facilities  and  dollars.  At  some  point  in 
time  any  one  of  these  resources  might  be  a  constraint  on  the  program.  However, 
since  this  is  a  budget  model,  it  considers  only  the  input  of  dollars.  A  still  more 
restrictive  definition  of  the  costs  considered  must  be  used,  however.  To  achieve 
the  value  determined  in  the  last  chapter,  a  system  must  be  developed,  produced 
and  operated  over  its  useful  life.  It  would  then  appear  obvious  that  the  relevant 
cost  considerations  .should  be  the  total  RDTE,  PEM.A  and  OMA  cost  ows, 


appropriately  discounted.^^  For  models  specifically  pertaining  to  project  selection, 
Le.,  those  dealing  only  with  the  decision  of  which  systems  should  be  in  the  program, 
all  of  these  dollar  costs  must  be  considered.  However,  since  this  model  is  designed 
to  investigate  RDTE  budget  alternatives,  only  RDTE  dollar  costs  are  considered 
directly.  Recau,  though,  that  survey  respondents  were  directed  to  consider  PEMA 
and  OMA  costs  in  th'  determination  of  value.  To  avoid  ambiguity  the  term 
“cost”  as  used  in  the  remainder  of  this  paper  will  refer  to  the  total  RDTE  dollar 
cost  for  a  system  unless  otherwise  specified. 

B.  EXPECTED  VALUE  OVER  TIME  FOR  A  FIXED  BUDGET 

What  does  an  incremental  change  in  the  RDTE  budget  of  a  system  buy  in 
terms  of  value?  Investigation  of  this  question  is  an  important  step  in  determining 
how  value  varies  with  cost. 

First,  if  the  performance  characteristics  (and  thus,  probably  the  value)  are 
upgraded,  an  increase  in  cost  is  likely  to  result.  Similarly  increasing  the  budget 
may  allow  the  value  to  increase,  either  from  an  upgrade  in  the  performance  character¬ 
istics  or  a  reduction  in  uncertainty  that  the  characteristics,  as  planned,  will  be 
achieved. 

A  second  way  that  value  might  vary  with  cost  is  in  the  time  that  is  likely  to 
be  required  in  the  development  program  before  the  system  reaches  operational 
readiness.  An  increase  in  the  budget  could  possibly  cause  a  shortening  of  the 
development  time,  while  a  reduction  might  require  that  the  program  be  stretched. 

^^While  discounted  costs  are  the  appropriate  consideration,  discounting  will 
be  disregarded  in  tJie  remainder  of  the  text.  It  could  be  easily  incorporated  into 
the  model. 

^^It  is  shown  later  that  considering  only  RDTE  costs,  rather  th  i  total  systems 
cost,  will  further  restrict  the  use  of  this  model  in  assisting  with  what  might  be  called 
the  ADD/DROP  decision. 
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To  determine  how  the  value  of  each  system  varies  with  RDTE  costs,  it  is 
necessary  to  gather  more  data  from  individuals  intimately  connected  with  the 
development  of  the  system.  To  do  this  we  might  survey  program  managers  with 
the  following  type  questions. 

You  are  the  program  manager  for  System  127,  Kickapoo  helicopter.  System 
127  is  composed  of  the  folloviing  projects  with  e  base  case  RDTE  funding  schedule 
as  shown. 


Project  FY  Budget  (in  Millions) 


Number 

Description 

73 

74 

75 

76 

77 

78 

79 

112 

Platform  Engr  Dev 

30 

30 

30 

163 

Platform  Adv  Dev 

50 

50 

50 

176 

Engine  Engr  Dev 

10 

10 

193 

Engine  Adv  Dov 

10 

10 

403 

Wpn  Sys  Dev 

20 

20 

20 

519 

Test  and  Eval 

5 

Total  Annual  Cost:  40 

40 

40 

80 

70 

70 

5 

Total  RDTE  Base  Funding  Cost:  3 15 


Question  1.  What  are  the  earliest,  latest,  aiid  most  likely  operational  readiness  dates 
(ORD)  at  this  base  funding  schedule? 

Question  2.  Consider  a  total  RDTE  Funding  level  ten  percent  greater  than  base. 
How  would  you  recommend  using  these  funds  to  change  performance  characteristics 
and/or  ORD?^^  Record  these  new  characteristics  and  the  earliest,  latest  and  most 
likely  ORD.  Also  record  the  rew  funding  schedule. 


24 

Each  major  hardw;ire  system  has  a  program  manager.  For  systems  defined 
in  Non-Hardware  Categories,  it  is  presumed  that  someone  has  the  necessary  famil¬ 
iarity  to  respond  to  such  a  sur\ey. 

r>r 

Note  that  the  program  manager  is  asked  to  give  his  opinion  ■  f  what  might 
be  called  the  technically  fcasii)le  and  optimal  tratle-off  of  performs  ■  characteristics 
for  ORD.  It  is  jjresumed  that  his  knowledge  of  the  it  chnologica!  i  .“rtainty 
leads  to  a  feasible  trade-eff.  Whether  or  not  it  is  the  "best”  tradeoif  is  a  matter  to 
be  ciuefully  itudied  by  users  of  the  model. 
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Question  3.  Answer  question  2,  but  consider  a  total  funding  level  20  percent  above 

base.^® 

Question  4.  At  what  total  RDTE  funding  level  would  the  Army  be  better  off 
dropping  this  development  program? 

Question  5.  Letting  Y  be  your  answer  to  question  4,  consider  a  funding  level  equal 
to  Y  +  1/3(345  -  Y).  Answer  question  2  at  this  funding  level. 

Question  6.  Consider  a  funding  ievel  of  Y  +  2/3(345  -  Y).  Answer  question  2  with 
the  total  budget  constrained  at  this  level. 

Note  that  question  4  identifies  a  “disaster  funding  level,”  at  or  below  which 
the  program  manager  feels  the  Army  is  better  off  dropping  the  system.  The  other 
questions  are  designed  to  collect  information  on  value  changes  and  readiness  date 
changes  for  five  funding  levels  above  disaster,  two  on  the  high  and  two  on  the  low 
side  of  the  base  case  funding  level. 

Note  also  that  for  any  of  the  four  levels  above  disaster  (excluding  the  base 
case)  for  which  the  program  manager  identified  significant  cha-.ges  in  performance 
characteristics,  it  would  be  necessary  to  treat  the  system  as  a  new  system  and 
re-survey  the  category  managers  for  different  value  parameters. 

Determining  the  earliest,  most  likely  anu  latest  operational  readiness  dates  is 
a  technique  from  PERT  theory  used  to  construct  a  probability  of  completion 
distribution  over  time.  The  three  parameters  are  usually  used  to  con«truct  a  Beta 
probability  distribution.  For  the  purposes  of  this  model,  however,  a  simple  linear 
cumulative  distribution  function  will  be  defined  as  shovn  in  Figure  1,  where  P(t) 
represents  the  probability  that  the  system  is  in  an  operational  readiness  state  in 

^®The  earliest  ORD  answer  to  this  question  defines  to  which  t'  e  period  the 
system  belongs. 
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year  t,  and  tQ,  and  tp  represent  the  optimistic,  most  likely  and  pessimistic 
completion  times. 


Figure  1.  CUMULATIVE  PROBABILITY  OF 

operational  readiness  over  time 


Tiie  one  final  set  of  data  required  for  the  model  is  indirectly  a  measure  of 
its  useful  life.  This  data  might  be  obtoined  by  again  surveying  the  category 
managers  with  a  question  I'ke,  “what  is  the  earliest,  most  likely  and  latest  years 
you  would  expect  this  system  to  be  retired  from  the  operational  inventory?” 

Since  it  is  poss-  that  this  probability  distribution  of  retirement  time  is  related 
to  the  operat'onal  readiness  dale  fand  therefore  to  the  funding  level),  th's  question 
should  probably  be  disked  for  each  funding  level  rmd  conditioned  on  the  distribution 
of  the  operational  roadir-ss  date  for  that  level. 

We  now  have  reached  the  point  where  enough  information  is  available  to 
construct  an  expected  value  versus  time  curve  for  the  i,  system  funded  at 
level  f,  where: 

f  =  1  Funding  level  Y,  “dbaster  funding” 

f  =  2  Y  -  1/3  (Base  -  Y) 

f  =  3  Y  +  2/3  (Base  -  Y) 

f  -  4  Base  case  funding 

f  =  5  Ten  percent  above  t)a.se  case 
f  =  6  Twenty  percent  al)ove  base  case 
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Figure  2  shows  the  typical  shape  of  the  curve  for  a  time  period  1  system. 


Figure  2.  EXPECTED  VALUE  OF  A  SYSTEM  OVER  TIME 
FOR  A  FIXED  FUNDING  LEVEL  "f" 


In  Figure  2,  the  highest  curve,  encompassing  both  dashed  and  solid  line  segments, 
re^^i^sents  the  value  of  a  system  when  availability  is  not  ccnsidered.  It  is  constructed 
using  a  linear  interpolation  between  the  value  degradation  points  determined  in  the 
category  managers’  survey.  The  solid  curve,  the  expected  value  of  t'ae  system,  is 
the  product  of  the  value  of  the  system  in  year  t  times  the  probability  the  system  is 
operational  in  year  t. 

For  each  system  a  curve  similar  to  Figure  2  would  be  defined  for  each  of 
funding  levels  2  through  6.  They  may  differ  for  each  funding  level  in  one  of  the 
following  ways. 

1.  If  ih“  performance  characteristics  changed  and  resulted  in  a  change  in 
the  value  parameter  for  the  system,  the  highest  curve  (value  curve  )  will 
be  shifted  up  or  down 
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2.  Different  distributions  of  the  probability  of  entry  and  exit  times  will 
undoubtedly  occur,  causing  a  shift  of  the  expected  value  curve  in  the 
uncertain  region  (the  years  where  the  probability  of  availability  is  less 
than  1)  to  the  right  or  left. 


C.  THE  transformation  TO  AN  EXPECTED  VALUE  VERSUS  COST  CURVE 
Since  a  curve  similar  to  Figure  2  can  be  constructed  for  each  system  i  for  each 
of  five  different  funding  levels  f,  a  method  is  avaUable  for  plotting  how  the  total  life 

expected  value  vanes  with  total  RDTE  funds.  Define  the  total  life  expected  value  of 

th  97 

the  i  system  when  funded  at  level  f  as  follows. 

where  t"  =  max  (the  optimistic  retirement  year)J 
i 

For  each  system,  apply  Equation  6  for  each  of  the  five  funding  levels  above  disast.er 
funding.  A  curve  similar  to  Figure  3  will  result. 


27 

For  simplicity  of  notation  the  category  subscript  j  will  oc  drc  uped.  The 
second  subscript  f  will  be  used  to  denote  which  of  the  (1  funding  K  '  -  is  being 

referred  to. 
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An  attempt  might  be  made  to  fit  an  analytical  curve  between  the  data  points, 
but  it  is  doubtful  that  much  would  be  gained,  and  it  would  result  in  a  difficult 
problem  to  solve  for  an  optimum. 

Although  Figure  3  shows  total  life  expected  value  to  be  a  simple  function  of 
total  RDTE  cost,  actually  it  is  quite  complex.  Incorporated  in  it  is  information 
concerning  the  system’s  value  relative  to  all  others  in  the  program,  its  expected 
useful  life,  technological  uncertainty  and  all  the  other  factors  that  have  been  con¬ 
sidered  up  to  this  point.^® 

Note  might  be  taken  of  the  shape  of  the  curve.  In  the  cost  range  [0,  f  =  6] , 
or  zero  to  20  percent  above  base,  it  has  a  characteristic  “S-shape”,  reflecting  no 
value  achieved  until  a  certain  “buy-in”  cost  is  paid.  In  the  cost  range  [f  =  1,  f  =  6] , 
value  is  shown  to  be  an  increasing  function  with  cost,  although  increasing  at  a 
diminishing  rate.  The  curve  is  said  to  be  conca'"*  in  the  region  [f  =  1,  f  =  6] .  Until 
such  time  as  actual  data  is  collected,  there  is  no  way  of  knowing  for  sure  that  such 
a  curve  will  result  for  each  of  the  systems.  To  assume  that  the  cun'e  is  concave  over 
the  region  of  positive  value  is  to  accept  the  widely  used  assumption  from  economic 
theory  of  diminishing  marginal  returns.  Simply  stated  this  says  that  as  funds  are 
increased,  a  dollar  buys  more  value,  but  not  as  much  more  as  the  dollar  before  it 
bought. 


^®AppondL\  A  lists  more  precisely  how  all  the  v.irial)Ies  are  fi  lionally 
related  to  TV-. 


IV,  FORMULATING  MATHEMATICAL  PROGRAMS  FOR 


THE  SIX  DEaSION  PROBLEMS 

Thus  far  the  discussion  has  dealt  primarily  with  defining  the  problem  and 
gathering  input  data  for  the  construction  of  total  life  expected  value  versus  cost 
curves  for  each  system  in  the  program.  In  this  chapter  the  discussion  centers  on 
how  this  information  is  used  in  mathematical  programs  to  provide  the  decision 
maker  with  insight  into  selecting  the  “right”  combination  of  the  six  options  he 
has  for  altering  the  RDTE  budget. 

A.  ALTERNATIVE  TIME  PREFERENCE  STRATEGIES 

Suppose  international  tensions  had  reached  the  state  where  it  was  increasingly 
apparent  that  World  War  ill  v/as  likely  to  begin  in  the  near  future.  Certainly  the 
Army’s  R  &  D  effort  would  emphasize  concentration  on  near-term  systems  to  the 
detriment  of  those  planned  for  operational  readiness  much  further  in  the  future. 
Conversely,  in  a  world  situation  where  war  seemed  highly  unlikely  in  the  near  future, 
the  “best”  strategy  might  be  to  concentrate  the  R  &  D  effort  on  the  later  systems, 
gambling  that  the  resources  saved  by  a  de-emphasis  on  near-tenn  systems  could  be 
better  spent  on  those  being  developed  for  the  out  years.  Lacking  certain  knowledge 
of  the  future,  there  is  no  “best”  strategy;  nor  should  the  ana.,  t  try  to  impose 
one.  This  is  a  strategy  decision  of  great  importance  that  should  be  made  only  at 
the  highest  levels. 

Recall  that  a  time  period  1  system  was  selected  as  the  category  numeraire  for 
each  category,  and  tl-.at  systems  from  later  time  periods  were  related  to  it  using 
the  time  degraded  value  of  the  numeraire.  In  essence,  then,  all  systems  are  measured 
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in  what  might  be  called  present  value.  This  was  purposely  done  to  allow  for  the 
input  of  alternative  strategies.  Loosely  define  three  different  strategies  in  the 
following  manner. 

Short  range  strategy:  Relatively  more  emphasis  is  placed  on  achieving  value  in 
the  near  term. 

Neutral  strategy:  The  importance  of  achieving  value  is  equal  in  all  time  periods. 

Long  range  strategy:  Relatively  more  emphasis  is  placed  on  achieving  value  in 
the  out  years. 

Any  of  these  strategies  can  be  incorporated  ito  the  model  in  the  following 
way.  Ask  the  user  to  assign  the  number  1  to  the  time  period  he  wishes  to  emphasize 
the  most.  Then  assign  fractional  numbers  to  the  other  periods  indicating  their 
importance  relative  to  the  most  important  one.  For  example,  parameters  for  a  short 
range  strategy  might  be  (1,  3/4,  1/2),  indicating  that  the  user  values  operational 
systems  twice  as  much  in  period  1  as  in  period  3.  Similarly,  parameters  for  a  long 
range  strategy  might  be  (8/10,  9/10,  1),  and  for  a  neutral  strategy  (1,  1,  1). 

Strategies  can  be  incorporated  into  the  model  by  using  a  time  preference 
param'.ter,  d^,  defined  as  shown  in  Figure  4  (for  a  short  range  strategy).  Note  that 
a  constant  rate  of  change  of  time  oreverence  is  assumed,  as  well  as  a  constant  time 
preference  for  the  years  not  between  the  data  points.  Once  the  user  identifies 
tne  three  parameters  reflecting  the  strategy  he  wants  to  investigate,  is  defined 
for  each  year  t.  The  expeettd  value  cvrrves  are  then  modified  to  reflect  the  importance 
the  user  places  on  ope;ational  systems  in  any  year  by  multiplying  the  expected 
value  in  yerr  t  times  d^.  The  result  is  what  is  commonly  called  discounted 
present  value. 
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Figure  4.  TIME  PREFERENCE  PARAMETER 

The  impact  of  inserting  a  time  preference  strategy  can  be  seen  in  Figure  5. 

The  solid  curves  represent  the  expected  value  as  determined  in  the  last  chapter. 

They  also  represent  a  neutral  (1,  1,  1)  strategy.  The  dashed  curves  represent  a 
short  range  (1,  L/4,  1/2)  strategy,  while  the  dotted  curves  are  a  (1/2,  3/4,  1)  long 
range  strategy.  Naturally,  since  strategies  alter  the  expected  value  curves,  the 
total  life  expected  value  versus  cost  curves  will  also  be  changed. 

Besides  the  obvious  partiality  for  systems  of  a  particular  time  period  that 
results  from  selecting  a  strategy,  strategies  also  cause  changes  in  the  relative  import¬ 
ance  of  certain  variables.  For  example,  a  short  range  strategy  would  lessen  the 
advantage  of  a  long  life  system,  while  a  long  range  strategy  lessens  the  importance 
of  an  earlier  operational  readiness  date  in  a  time  period  1  system.  All  of  these 

factors  are  incorporated  into  the  model  by  the  simple  multiplication  of  d^  •  E[Vj]  , 

t 

B.  DETER.MINING  THE  OPTIMAL  PROGRAM  FOR  A  GIVEN  STRATEGY 
AND  BUDGET 

Once  a  strategy  has  been  .selected  by  the  user,  a  specific  TVj  versus  cost 
curve  is  identified  for  each  .system.  Suppose  the  u.ser  wishes  to  use  (  \RDBOMB 
to  determine  an  optimal  RDTE  program  for  a  given  total  RDTE  bui.^et  constraint 
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for  FY  73  of  B  billion  dollars.^^  He  desires  that  this  be  determined  without  con- 

on 

sidering  dropping  or  slipping  any  of  the  systems  currently  in  the  base  case  program. 
Since  dropping  a  system  is  not  to  be  considered,  attention  can  be  directed  to  the 
TVj  curves  in  the  cost  range  above  f  =  1,  the  disaster  funding  level.  That  is, 
only  funding  levels  2  through  6  are  to  be  considered  feasible  alternatives  for  each 
system.  With  the  assumption  that  the  values  of  the  systems  are  independent  of 
each  othcr^the  mathematical  program  can  be  formulated  as  follows: 

n 

Maximize:  ^  TVj(xj) 

i=l 

n 

Subject  to:  51  73  —  B73 

where  xj  =  the  total  RDTE  cost  of  the  i^^  system 

Cj  f  ^  =  the  year  t  cost  of  the  i*'"  system  when  funded  at  level  f. 
Unfortunately  (in  a  mathematical  sense)  the  budget  constraint  is  on  the  FY  73 
costs  and  not  on  the  total  RDTE  costs.  While  it  may  be  likely  that  the  TV|(X|) 
curves  are  concave,  it  is  possible  that  some  of  the  TVj  versus  year  t  costs  eire  not. 

As  an  illustration  consider  the  two  curves  in  Figure  6. 

The  right  curve  illustrates  the  case  where  year  t  costs  as  estinu.ted  by  the 
program  manager  are  higher  at  funding  level  3  than  they  are  at  f  =  1,  certainly 
a  possibility  for  some  systems.  Solving  for  an  optimal  solution  when  the  curves 


billion  would  represent  the  total  funds  allocated  to  RDTE  minus  the 
amount  reciuired  for  tlie  categories  of  Basic  Research  and  Management  and  Support. 
These  are  still  being  excluded  from  the  discussion.  Although  the  fir  t  year  is  being 
used  as  the  budget  year  under  investigation,  the  model  can  also  be  used  to  investi¬ 
gate  constraints  in  years  other  than  this  first  year.  Discussion  of  this  point  is 
deferred  until  Chapter  5. 

^^Fhesc  two  decision  options  are  discussed  in  Sections  D  and 

^^The  implications  of  this  as.sumption  are  discusscil  in  Sectii  of  this  chapter 
and  in  AppendLx  A. 
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in  the  objective  function  are  not  concave  can  be  a  difficult  task  involving  complex 
search  procedures.  However  the  following  simple  algorithm  (solution  method)  will 
lead  to  an  approximation  to  the  optimal  solution. 

Algorithm  1 

1.  Calculate  and  label  the  slopes  of  the  four  linear  segments  of  the  TVj  versus 
total  RDTE  cost  curve  above  funding  level  f  =  2  as  (i,  s),  for  i  =  1,  .  .  .  ,  n 
systems  and  s  =  1,  ....  4  slopes.  These  slopes  are  measures  of  marginal 
value  between  the  funding  levels. 

2.  Order  these  4  n  slopes  in  decreasing  sequence,  K  =  1,  .  .  .  ,  4n. 

3.  Calculate  the  initial  value  of  a  running  sum 

n 

*^o  "  ^i,2,73  ’ 

i=l 

the  totn!  FY  73  cost  of  the  program  when  ail  systems  are  funded  at 
f  =  2.  Set  K  =  1. 

4.  Identify  the  i^*^  system  belonging  to  slope  K.  Calculate  the  increase 
in  FY  73  costs  for  this  system  when  it  is  funded  one  level  higher. 

Qk  “  ^i,f+l,73  •  ^i,f,73- 

5.  If  +  Qk  ^  B,  go  to  step  8. 

6.  If  R^-i  Qk  ^  ®  ~  ^K"l  funding  level  of 

the  i^^  system  to  f  =  f  +  1. 

7.  Set  K  =  K  +  1  and  return  to  Step  4. 

8.  Stop.  An  approximation  to  the  optimum  has  been  reache 

09 

•‘How  close  lliis  approximaiion  is  to  the  optimum  is  discu^si  .  Appendix  A, 
as  well  as  more  elaborate  procedures  for  finding  the  actual  optimum. 
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A  simple  example  of  this  algorithm  may  help  to  clarify  its  use.  Suppose  the 
RDTE  program  consisted  of  two  systems  whose  total  value  versus  cost  curves  were 
as  shown  below.  Additionally,  the  program  managers  for  these  two  systems  had 
estimated  the  total  FY  73  funding  for  each  of  funding  levels  2  through  6  to  be: 

Funding  Level 

I 

r  3  4  5  6 

«  ,  1  (Is  30  35  40  45 

System  2  60  70  80  90  100 


The  slope  ordering  list  called  for  in  step  2  of  the  algorithm  is  reflected  in  the 
third  column  of  the  “Budget  Incrementing  Matrix”  shown  in  Figure  7.  Note 
Rq  »  25  +  60  =  85.  If  tho  budget  constraint  being  investigated  were  B  =  115, 
the  algorithm  would  stop  at  K  =  4,  reflecting  both  systems  funded  at  their  base 
funding  levels.  The  total  value  accruing  to  the  program  would  be  20  +  19  =  39. 


System 

f 

Marginal 

Value 

FY  73 
Funds 
Added 

R 

K=  1 

1 

3 

0.7 

30-25 

90 

2 

2 

3 

0.6 

70-60 

100 

3 

1 

4 

0.5  » 

35-30 

105 

4 

2 

4 

0.4 

80-70 

115 

-  5 

1 

5 

0.3 

40-35 

120 

6 

2 

5 

0.2 

90-80 

130 

7 

1 

6 

0.15 

45-40 

135 

8 

2 

6 

0.1 

100-90 

145 

Figure  7.  BUDGET  INCREMENTING  MATRIX 
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As  has  been  mentioned,  once  a  strategy  has  been  selected  a  TV^  versus  cost 
curve  is  identified  for  each  system.  Additionally,  a  budget  incrementing  matrix 
is  defined.  If  a  user  wished  to  investigate  some  alternative  RDTE  budget  level 
for  FY  73,  the  recommended  changes  from  the  base  case  are  clearly  visib’e  in 
the  Budget  Incrementing  Matrix.  For  example  if  he  wished  to  investigate  a  con¬ 
straint  of  B  ”  135,  the  model  would  recommend  he  consider  raising  system  1  to 
f  =  6  and  system  2  to  f  “  5. 

Studying  Figure  7  should  make  the  need  for  the  assumption  of  corkcave  curves 
apparent.  If  at  some  point,  diminishing  marginal  value  was  not  the  case  for  a 
particular  system,  then  the  slope  sequence  list  might,  for  example,  call  for  an 
increase  to  f  =  5  before  f  =  4  was  achieved.  For  systems  that  might  exhibit  a 
“violation”  to  the  law  of  diminishing  marginal  value,  the  following  procedure  can 
be  applied  as  a  modification  to  Algorithm  1.  Consider  a  system  whole  TVj  versus 
cost  curve  is  as  shown  in  Figure  8.  For  some  reason  little  increase  in  value  is 


Figure  8.  NON-CONCAVE  VALUE  CURVE 

achieved  in  going  from  f  »  3  to  f  •=  4  (marginal  value  near  zero).  The  dotted  lino 
between  f  ■  3  and  f  “  5  docs  represent  the  marginal  value  between  t'  ese  points, 
however.  The  approximation  to  the  optimal  solution  cuul.i  be  oht.  d  by  using 


the  slope  of  this  interfiolated  line  in  Algorithm  1.  If  it  resulted  in  f  =  2,  3  or  6 
being  recommended  for  this  system,  the  non-concave  portion  of  the  curve  had  no 
impact.  If  f  =  Ji  were  selected,  however,  it  would  be  necessary  to  check  whether 
or  not  reducing  this  system  to  f  =  4  would  free  funds  to  be  spent  on  another 
system  which  would  result  in  greater  program  value. 

0,  NESTING  THE  SOLUTION  AGAINST  THE  BASE  CASE  PROGRAM 

Suppose  Algorithm  1  were  used  with  an  FY  73  budget  constraint  equal  to  that 
planned  for  the  base  case  program.  Undoubtedly  a  solution  other  than  the  base 
case  would  result,  i.e.,  some  systems  would  be  funded  higher  than  base  funding 
(f  =  4)  and  some  would  be  lower.  This  could  result  from  -my  of  the  following 
reasons. 

1.  The  time  preference  paramett.  use;  '  did  not  adequately  reflect  the 
actual  time  preferences  of  the  collective  Icadersliip  of  DA. 

2.  Some  of  the  value,  need,  useful  life  or  other  parameters,  all  consensus 
measures  determined  by  advisors  or  program  managers,  do  not  reflect 
tlie  views  of  the  decision  makers. 

3.  The  base  case  program  contains  inefficiencies,  i.e.,  after  more  careful 
analysis,  a  better  program  might  result  if  funds  were  taken  from  some 
systems  and  given  to  others. 

4.  The  appro.\imation  to  the  optimal  solution  determined  by  .Alogrithm  1 
was  not  close  to  opti../'^ 

5.  Some  combination  of  two  or  more  of  the  above  reasons. 

Resolving  the  differences  between  the  solution  as  determined  by  the  algorithm 
and  the  the  actual  base  case  program  would  be  a  difficult  but  useful  exercise. 

First,  it  might  identify  inefficiencies  in  the  actual  base  case,  suggesting  a  better 
distribution  of  funds.  Second,  if  the  model  is  to  be  of  any  use  at  all  in  investigating 
alternative  budget  constraints,  tiie  parameters  need  to  reflect  a  gene  1  agreement 
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on  thsir  validity.  An  investigation  of  the  dive  rgence  of  the  solution  and  the  base 
case  might  take  the  fohowing  torm. 

First  an  attempt  to  resolve  the  tiino  preference  parameters  to  see  if  some 
particular  set  comes  closest  to  producing  a  solution  similar  tc  the  base  case.  For 
example,  starting  witl)  .1  neutral  strategy  (1,  1,  1),  move  increasingly  in  the  direction 

00 

of  a  short  ran,-,. 3  strategy.*^^  The  criteria  for  a  set  of  strategy  parameters  being  more 
suitable  than  another  could  be  the  number  of  systems  funded  at  levels  other  than 
base  funding.  As  an  example,  suppose  this  investigation  resulted  in  the  following. 

Time  Preference  Number  of  Systems 

Parameters  Not  at  f  =  4,  Base  Funding  Level 

(1,  1,  1)  25 

(1,  .95,  .9)  20 

(1,  .9,  .85)  15 

(1,  .85,  .80)  6 

(1,  .8,  .7)  10 

(1,  .7,  .6)  20 

It  might  then  be  assumed  that  (1,  0.85,  0.8)  most  closely  reflect  the  time  preference 
of  DA  decision  makers.  Of  course  there  is  no  way  to  prove  that  these  parameters 
are  “correct”,  but  tlioy  do  imply  that  DA  decision  makers  have  acted  as  if  these 
were  their  time  preferences. 

The  investigation  might  then  continue  with  a  careful  analysis  of  ‘he  systems 
not  at  funding  level  4.  This  investigation  might  result  in  agreement  that  the  responses 
to  the  surveys  were  unrealistic  in  some  cases  and  some  of  the  input  parameters  need 
adjusting.  Alternatively  it  might  be  agreed  that  the  base  case  would  indeed  be 
better  if  some  redistribution  of  funds  were  made. 

®^Due  to  the  nature  of  national  defense,  defense  decision  makers  are  generally 
presumed  to  prefer  forces  in  being  to  systems  planned  for  further  in  the  future. 
Starting  the  investigation  from  a  neutral  strategy  and  moving  towarii  strategics 
which  are  more  and  more  short  range  should  mir.imize  tne  number  1  ;  sets  of 
parameters  required  for  this  investigation. 
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D.  IDENTIB'YING  SYSTEMS  TO  BE  SLIPPED 


The  last  two  sections  dealt  with  the  problem  of  determining  an  optimal  RDTE 
program  for  a  given  fiscal  year  budget  constraint.  The  method  formulated  assumed 
that  the  user  did  not  wish  to  consider  slipping  a  system’s  development  schedule 
one  or  more  years.  As  has  been  stated,  slipping  consists  of  delaying  the  start  of 
a  development  program,  or  interrupting  one  that  has  already  begun  (such  as  between 
the  engineering  development  and  advanced  development  stages  with  the  intent  of 
starting  it  up  again  at  a  later  date).  This,  of  course,  is  a  decision  option  open  to 
the  Army  for  altering  the  RDTE  budget  program  for  rny  given  year.  We  now 
consider  how  the  model  can  assist  the  usei  in  identifying  systems  which  might  be 
slipped  one  or  more  years. 

The  first  consideration  that  must  be  addressed  is  whicti  systems  might 
reasonably  be  considered  for  slipping.  Certainly  if  one  or  more  of  the  projects 
of  a  system  scheduled  for  FY  73  are  “carry-overs”  from  FY  72,  it  might  be 
unreasonable  to  expect  that  it  would  be  economical  to  interrupt  work  on  those 
projects  for  one  or  more  years  and  then  begin  them  again.  This  fact  alone  will 
undoubtedly  eliminate  many  systems  from  consideration  for  slippini’. 

One  must  also  consider  what  might  be  gained  or  lost  from  slipping  the 
schedules  of  systems.  The  first  “gain”  might  be  called  a  reduction  in  technological 
uncertainty.  The  development  cycles  of  systems  follow  a  general  sequential  pattern 
starting  with  the  concept  development  stag^  and  progressing  through  engineering 
development,  advanced  development,  test  and  evaluation,  aaid  finally,  production 
and  operational  systems  development.  Between  each  of  these  stages  of  development 
a  detailed  review  of  the  progress  that  has  been  achieved  must  be  mm'e  and  a 
decision  issued  on  whether  or  not  to  begin  the  next  st.tge.  It  might  ■  determined 
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that,  in  order  to  overcome  technical  problems  which  have  not  been  resolved,  the 
best  course  of  action  is  to  slip  the  next  stage  one  or  more  years.  The  decision 
problem  associated  with  slipping  a  system’s  development  schedule  to  over^’orne 
technical  difficulties  is  not  addressed  in  this  model.  It  is  assumed  that  these  decisions 
have  already  been  made  in  arriving  at  the  base  case  program. 

Besides  reducing  technological  uncertainty,  the  Army  might  also  wish  to 
consider  slipping  the  development  schedule  of  one  or  more  systems  to  save  funds 
in  a  particular  year.  The  model  can  assist  with  this  decision  option  by  identifying 
a  measure  of  the  value  lost  per  dollar  saved  when  a  system  is  slipped. 

Suppose  that  for  a  particular  time  strategy  and  total  budget  constraint, 
Algorithm  1  resulted  in  funding  level  f  being  optimal  for  the  i^*'  system.  The 
expected  value  versus  time  curve  for  this  system  at  this  funding  level  might  be  as 
shown  by  the  solid  line  in  Figure  9.  Under  the  assumptions  that  (1)  distribution  of 
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FIgura  9.  LOSS  IN  VALUE.  L,.  CAUSED  BY  A  ONE  YEAR  SLIP 

the  probability  that  a  system  is  operational  by  year  t  is  affected  by  a  one-year 
schedule  slip  only  to  the  extent  that  it  is  displaced  one  year,  and  (2)  the  probability 
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distxibution  of  the  system  retirement  date  is  not  affected  by  a  one-year  slip,®^  then 
the  loss  in  value,  L|,  caused  by  a  one-year  slip  to  the  development  schedule  of  the 
system  can  be  approximated  by; 


M - -  m 

Dividing  Lj  by  the  FY  73  budget  for  the  i^*'  system  Cj  f  (73,  would  then  determine 
an  approximation  to  the  loss  in  value  per  dollar  saved  by  a  one-ycar  slip.  This 
presumes  that  no  money  would  be  required  to  keep  the  program  "alive”  during  the 
slip.  If  this  were  not  the  case,  Cj  f  -13  would  repre.sent  the  difference. 

If  a  user  of  the  model  wished  to  consider  slipping  as  an  alternative  for 
investigating  different  RDTE  programs,  this  could  be  incorporated  in  the  following 
way. 


Algorithm  2 


1.  Use  Algorithm  1  to  determine  the  approximation  to  the  optimal  RDTE 
funding  level  f  for  each  system  when  slipping  is  not  considered. 

2.  For  the  subset  of  systems  identified  as  candidates  for  slipping,  calculate 
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the  loss  in  value  per  FY  73  dollar  saved. 

3.  Order  the  set  of  (sjj^  in  increasing  sequence  k  =  1,  .  .  .  ,  n.  (The  system 

with  the  smallest  loss  in  value  per  dollar  saved  is  first  in  this  sequence, 
k  =  1).  Set  k  -  1. 

4.  Identify  the  k^^  system  as  the  next  system  in  the  budget  incrementing 
matrix  to  be  increased  if  more  FY  73  funds  were  available.  Calculate 


^^An  alternative  to  these  assumptions  would  be  to  gather  more  lata  on  how 
a  one  year  sliji  would  affect  the  earliest,  latest,  and  most  likely  oper  ional  readiness 
dates  and  system  retirement  dates. 
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TVk.fn  •  TVk,f 

Ck,f+1,73  •^k,f,73 


the  gain  in  total  value  to  the  k^^  system  per  FY  73  dollar  added, 

5.  If  G|^  (if  the  loss  in  total  value  caused  by  slipping  system  i  one 

year ’is  greater  than  the  gain  in  total  value  to  system  k  when  the  money 
saved  by  the  slip  is  transfered  to  system  k),  go  to  step  8, 


6.  slip  system  i  one  year  and  transfer  the  savings  to  system  k. 

7.  Let  k  =  k  +  1  and  return  to  Step  4. 

8.  Stop.  Slipping  any  more  systems  and  transferring  the  funds  to  other 
systems  will  only  cause  a  net  decrease  in  the  total  value  of  the  RDTE 
program. 


E.  IDENTIFYING  WHICH  SYSTEMS  MIGHT  BE  DROPPED 

Another  alternative  for  altering  the  RDTE  program  is  to  drop  one  or  more 
systems.  As  has  been  mentioned,  this  decision  problem  is  normally  addressed  on 
a  system  by  system  basis  during  the  extensive  progi  ■  s  review  following  completion 
of  a  stage  of  the  development  cycle,  or  when  some  threshold  parameter  of  the  DCP 
has  been  violated.  It  could  also  be  addressed  on  a  total  program  basis,  i.e.,  given 
a  particular  FY  73  budget  constraint,  would  more  total  value  accrue  to  the  RDTE 
program  if  one  or  more  systems  were  dropped  and  the  savings  transl  orred  to  other 
systems?  It  has  previously  been  stated  that  CARDBOMB  can  only  indirectly  assist 
in  this  decision  problem  for  reasons  which  can  be  summarized  as  follows. 

The  interdependence  of  the  value  parameters.  As  has  been  mentioned,  the 
relative  value  of  the  i^^  system  is  a  consensus  measure  of  a  group  of  experts  who 
were  not  asked  to  envision  a  future  where  the  i^^  system  was  the  only  one  in  the 
inventory.  In  other  words,  the  value  of  this  system  depends  on  the  value  which 
results  in  other  systems.  The  optimization  procedure  discussed  thii-  r  has  ignored 
this  interdependence  and  assumed  that  the  value  achieved  by  one  system  does  not 
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alter  the  value  of  another.  This  independence  of  value  assumption  says,  for  example, 
that  the  change  in  expected  operational  readiness  date  and  performance  characteristics 
caused  by  a  change  in  the  funding  level  of  the  i^^  system  in  no  way  affects  the  value 
of  other  systems.  For  changes  in  value  in  the  region  around  the  base  case  funding 
level,  i.e.,  plus  or  minus  20  percent  of  base  funding,  this  assumption  is  probably  not 
a  bad  approximation  for  most  systems.  This  situation  will  undoubtedly  not  be  the 
case  when  dropping  a  system  is  considered.  While  moving  up  the  operational 
readiness  date  of  Tank  System  A  one  year  probably  does  not  significantly  affect 
the  value  of  Tank  System  B,  dropping  System  A  would  undoubtedly  have  a 
significant  impact  on  the  value  of  System  B.  Models  have  been  developed  which 
take  into  consideration  the  interdependence  of  value.^^  In  general  they  would 
require  a  far  greater  data  collection  effort  and  complex  dynamic  programming 
solution  methods. 

The  interdependence  of  value  and  total  system’s  cost.  The  surveys  to  determine 
the  value  parameters  asked  the  respondents  to  consider  PEMA  and  DMA  costs  in 
arriving  at  a  measure  o*'  a  system’s  value.  If  all  other  considerations  were  equal, 
a  -system  with  high  PEMA  and  OMA  costs  would  be  rated  lower  than  a  cheaper 
system  This  technique  was  used  to  insure  that  these  important  cost  considerations 
were  included  in  the  model,  but  not  on  the  cost  axis  of  the  TVj  versus  cost  curves, 
since  the  model  is  designed  primarily  for  investigating  only  the  RDTE  budget.  In 
eddressing  the  problem  of  which,  if  any,  systems  could  be  dropped,  the  appropriate 
measure  would  be  the  total  value  (measured  independent  of  ail  cost)  per  total 
system’s  cost  dollar. 


35 


See  Weingurtner  (ref.  9)  for  a  survey  of  these  models. 
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Keeping  in  mind  these  considerations,  CARDBOMB  still  might  provide  insight 
into  the  decision  problem  of  which  systems  might  be  dropped.  Using  the  total 
life  expected  value  for  the  i^'^  system  determined  using  algorithms  1  and  2,  divide 
this  by  the  total  expected  system’s  cost.  That  is,  for  each  system  i,  calculate 

TV? 

Di  = - !— - 

*i  +  yi-^Z  Pt^i 
t 

where 

0 

TVj  =  total  life  expected  value  at  the  optimal  funding  level, 
xj  ■=  total  RDTE  cost  at  the  optimal  iunding  level 
yj  =  total  PEMA  cost. 

Zj  =  annual  OMA  cost. 

P|.  =  as  before,  the  probability  the  system  is  operational  in  year  t. 

Ordering  the  set  of  Dj  in  increasing  sequence,  while  certainly  not  establishing  a 
priority  list  for  dropping  systems,  could  assist  the  Army  by  focusing  attention  on 
which  development  programs  might  be  more  carefully  analyzed.  The  first  system 
in  the  list  would  be  the  one  with  the  lowest  average  value,  or  value  per  total  dollar. 

F.  ANALYZING  THE  IMPACT  ON  THE  TOTAL  PROGRAM  OF  ADDING 
A  NEW  SYSTEM 

By  the  time  a  new  system  has  finished  the  concept  development  stage, 
considerable  cost-effectiveness  analysis  has  been  conducted  on  alternative  approaches 
to  meeting  the  threat  for  which  it  is  d.!signed.  Besides  these  analyses  a  necessary 
input  to  the  DCP  is  the  impact  the  introduction  of  this  system  will  have  on  other 
Army  programs,  i.e.,  what  must  be  given  up  to  get  it. 


64 


As  was  the  case  in  the  question  of  which  systems  might  be  dropped,  CARDBOMB 
is  limited  in  assisting  with  this  problem  for  much  the  same  reasons  —  the  inter¬ 
dependence  of  the  value  parameters  and  the  treatment  of  PEMA  and  OM  A  costs  as 
negative  value  rather  than  independent  variables.  However  insight  to  this  problem  still 
might  be  gained  in  the  following  way. 

As  was  done  for  systems  already  in  the  program,  survey  category  managers  to 
determine 

Vi 

V’ 

a  measure  of  the  value  of  this  new  i^^  system  on  the  same  scale  as  all  the  other 
systems.^^  Similarly,  gather  data  for  four  alternative  funding  levels  to  arrive  at 
a  TVj  versus  RDTE  cost  curve.  Include  this  new  curve  in  the  program  and,  using 
Algorithm  2  and  the  same  total  budget  constraint,  solve  for  an  optimal  solution. 

This  new  solution  will  determine  two  things.  First,  it  will  recommend  at  which  of 
the  five  funding  levels  this  system  ought  to  be  developed,  and  second,  the  impact 
on  the  systems  whose  funding  levels  had  to  be  lowered  determine  a  measure  of 
what  must  be  sacrificed  in  the  RDTE  program  to  get  this  system.  That  is, 
introducing  Syste.m  Z  into  the  program  might  require  reducing  the  design  character¬ 
istics  of  System  Y  and  stretching  the  operational  readiness  date  of  System  W  one 
year.  Alternatively,  the  Army  might  decide  to  maintain  the  design  cliaracteristirs 
of  System  Y,  raire  the  RDTE  budget  constraint  and  make  up  the  difference  in  some 
other  budget  program.  In  any  event  CARDbOMB  has  assisted  by  identifying  in 
terms  other  than  dollars,  what  must  be  given  up  to  get  System  Z. 

^®A  way  of  avoiding  the  interdependence  of  value  problem  in  s  case  is  to 
survey  to  determine  new  relative  value  p.arameters  for  all  the  syNte’:  .n  this 
category  when  the  new  system  is  considered  part  of  this  category. 


Of  coarse  these  measures  of  what  must  be  given  up  only  consider  the  RDTE 
budget.  Lacking  an  optimization  model  which  encompasses  the  RDTE,  PEMA  and 
OMA  budgets,  we  must  resort  to  the  average  value  methods  of  the  previous  section. 
Therefore,  besides  conducting  the  analysis  just  discussed,  it  might  also  be  appropriate 
to  determine  where  the  Dj  for  this  system,  the  total  value  per  total  systems  dollar^ 
fits  into  the  sequence  of  discussed  in  the  previous  section.  If  it  turned  out  to 
be  high  on  the  list,  meaning  it  has  one  of  the  lower  average  values  in  the  program, 
this  could  imply  that  it  niight  not  be  worth  the  cost  and  should  be  studied  fiurther. 

G.  BUDGETING  PROJECTS  WHICH  ARE  NOT  COMPONENTS  OF  A  SYSTEM 

DEVELOPMENT  PLAN 

The  discussion  of  the  model  has  thus  far  ignored  projects  which  are  not 
related  to  a  specific  system’s  development  plan.  In  the  classification  scheme  of 
this  paper,  those  projects  fall  into  Category  22,  Researcli,  and  Category  21, 
Management  and  Support.  We  now  turn  our  attention  to  theje  two  categories. 

The  inputs  required  for  an  optimization  model  must  answer  two  important 
questions;  what  is  the  value  of  the  system,  and  how  does  this  value  vary  with  cost. 
Undoubtedly  by  the  end  of  Chapter  3  the  reader  had  developed  an  appreciation 
for  the  difficulty  in  providing  quantitative  answers  to  these  questions.  The  problem 
is  even  more  difficult  when  considering  basic  research  and  exploratory  de■.^,l•Jp:nent 
projects.  Consider  a  hypothetical  project  designed  for  investigatinr*  military 
applications  of  a  rotary  engine.  Unlike  a  system  with  a  development  plan  which 
identifies  expected  performance  characteristics,  the  threat  it  is  designed  to  meet  and 
other  factors,  just  what  might  result  from  this  project  is  highly  uncertain.  Equally 
uncertain  is  how  a  funding  change  will  affect  the  results  of  the  project.  For  these 
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reasons  the  model  will  treat  category  22  projects  differently  than  it  has  treated 
V! 

systems. 

Define  twenty  sub-categories  of  category  22  to  correspond  with  the  classification 
scheme  for  systems,  i.e.,  sub-category  1  would  be  basic  research  projects  associated 
with  Air  Mobility.  Further  define  sub-category  21,  Pure  Research,  consisting  of 
all  projects  (excluding  Management  and  Support)  not  otherwise  classified.  Using 
the  methods  of  Chapter  2  for  determining  va^ue  parameters  of  systems,  obtain  a 
parameter  for  each  of  the  Category  22  projects.  That  is,  for  each  of  the  projects 
within  the  j^^  sub-category,  obtain  a  consensus  measure. 


of  the  relative  value  of  what  is  expected  to  result  from  the  projects.  Then  use 
these  with  either  the  primary  or  secondary'  methoas  of  Chapter  2  for  determining 
the  value  of  any  sub-category  project  relative  to  the  value  of  the  Category  22 
numeraire, 


Ideally  we  would  now  like  to  know  how  this  value  varies  with  funds  so  that 
measures  of  marginal  value  could  be  obtained.  Unfortunately  basic  research  projects 
do  not  lend  themselves  to  this  measuremcjnt  as  well  as  system  do.  While  an  increase 
in  funds  might  result  in  completing  the  project  sooner,  in  general  this  advantage 
would  be  impossible  to  measure  in  terr...s  of  changes  in  value.  However,  under  the 
assumption  that  for  small  funding  changes  near  base  case  funding,  the  ratio  of  the 

^"^For  a  more  detailed  discussion  of  considerations  pertinent  to  i  lodeling  basic 
research,  see  Quinn  (ref.  10] 
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marginal  values  of  two  projects  are  directly  proportional  to  the  ratio  of  their 
average  values,  we  can  develop  a  method  for  investigating  alternative  basic  research 
budgets. 

We  must  first  determine  a  method  for  measuring  the  value  of  the  i^^  project^^ 
per  total  KDTE  dollar,  being  spent  on  it,  defined  as 


the  average  value.  Recall  that  on  an  interval  scale  of  measurement,  the  particular 
value  assigned  to  the  numeraire  has  no  significance.  Arbitrarily  let  Vg  =  Xg.  We 
tiien  have 


V.  V-  V  V  X 

^  i  -  ll .  _3—  _L  3 

^  V.  V  X*  V  X- 

*1  *1 


(9) 


Note  that  on  this  arbitrarily  selected  scale,  the  value  per  dollar,  or  average  value,  for 
the  numeraire  project  reduces  to  Ag  =  1.  Another  project  which  '  ad  only  half  tlie 
value  of  the  numeraire  but  was  only  one-tenth  ;is  expensive  would  have  Aj  =  5. 

Before  turning  to  the  development  of  a  decision  rule,  let  us  examine  the 
implications  of  the  assumption  stated  above.  Figure  10  shows  two  basic  research 


Figure  10.  IMPLICATIONS  OF  THE  AVERAGE  VALUE  ASSUMPTION 

®®The  sub-category  subscripts  j  are  now  dropped  for  simplicity  oi  .tation. 
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projects  each  funded  at  a  total  RDTE  cost  of  xj  =  xj^..  Their  values  at  this  base  funding 
were  measured  at  1  and  Vi  respectively.  Their'  average  values,  or  value  per  dollar,  are 
then  measured  t:,  the  slope  of  the  dashed  line  from  the  origin.  The  solid  curves 
are  hypothetical  value  curves  which,  as  has  been  mentioned,  are  not  known.  Since 
they  are  unknown,  so  is  the  marginal  value  at  this  funding  level.  If  they  were  known 
we  would  want  to  increase  the  one  with  the  greatest  marginal  value  (or  decrease 
the  one  with  the  smallest).  Figure  10  shows  that  adding  a  dollar  to  the  budget  of 
project  k  would  cause  a  greater  increase  in  value  than  adding  it  to  i,  even  though 
project  i  has  the  greater  average  value.  This  discussion  is  added  to  demonstrate  how 
easily  the  assumption  can  be  violated;  the  ratio  of  average  values  are  not  always 
proportional  to  the  ratio  of  mai'ginal  values.  However,  lacking  information  on 
the  curve  itself,  it  might  be  further  assumed  that  the  base  case  funding  is  near  the 
point  where  value  is  “falling  off”,  as  is  the  case  in  Ihe  left  figure.  !n  other  words 
the  project  has  reached  the  point  where  hiring  a  few  more  scien;ists  just  isn’t  going 
to  add  that  much.  With  this  further  assumption,  we  not  continue  with  an 
algorithm  for  altering  the  Category  22  budget. 

We  wish  to  insure  tnat  if  the  budget  gets  increased,  the  projects  with  higher 
average  values  receive  proportionally  more  of  the  increase  than  those  with  lower 
average  values.  Conversely,  if  the  budget  is  reduced,  the  lowest  average  value 
projects  get  reduced  the  most.  These  “decision  rules”  are  depicted  in  Figures  11 
and  12,  where 

A'  =  max  Aj  and  A”  =  min  Aj 

i  i 

Suppose  a  user  of  the  model  wishes  to  investigate  the  Categc-ry  buoget 
program  that  would  result  from  using  this  method  for  a  tcial  consti  int  of  D  million 
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Figure  11.  COMPUTING  A  PERCENTAGE  INCREASE  FOR 
BASIC  RESEARCH  PROJrCTS 


decrease 

Figure  12.  COMPUTING  A  PERCENTAGE  DECREASE  FOR 
BASIC  RESEARCH  PROJECTS 


dollars,  less  than  the  base  case  program  total  constraint  of  E  million.  The  following 
formula  calculates  the  percentage  reduction  to  each  project  that  will  be  required  to 
meet  this  new  const'll. 


(A'  -  Aj)  (E  -  D) 
■  Z  (A-  .  A,)  C| 


(10) 
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To  illustrate  the  use  of  this  formula,  consider  a  three  project  basic  research 
program  with  average  value  and  FY  73  base  funding  costs  as  follows. 

Aj  -  A'  -  6  Cj  -  10 

A2  ■  3  C2  “  20 

Ag-A"-!  C3-4O 

Note  that  the  total  base  case  funding,  E  -  10  20  +  40  ■  70.  Suppose  the  user 

wished  to  investigate  a  constraint  of  D  ■  60.  Equation  10  would  result  in  the 
required  percentage  reduction  for  each  project  as  shown. 


_ (5-5)  (70-60) _ _ 

(5-5)  (10)  +  (5-3)  (20)  +  (5-1)  (40)  "  200  ~ 


(5-3)  (70-60)  1 

g  - - -  —  =10  percent 

^  000  in  ^ 


It  should  be  apparent  that  no  optimization  b  involved  in  this  method  for  altering 
the  budgets  of  Category  22  projects.  What  is  reflected  in  the  solution,  however,  is 
that  projects  with  high  value  per  total  dollar  figures  are  increased  proportionally 
more  and  decreased  less  when  the  total  Basic  Research  budget  Ls  alt  ,  d.  Of  course 
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the  resulting  solution  would  have  to  be  carefully  investigated  to  insure  that  the 
changes  could  be  absorbed. 

To  summarize,  the  decision  rule  for  altering  the  budgets  of  individual  Category 
22  projects  is:  If  the  budget  constraint  for  Category  22  projects  is  greater  (less) 
that  the  base  case  constraint  by  an  amount  D  -  E  (E  -  D),  then  increase  (decrea.se) 
each  project  the  percentage  of  its  base  case  funding  as  determined  by  Equation  11 
(Equation  10). 

Studying  changes  to  the  budgets  of  projects  in  Category  21,  Management 
and  Support,  is  not  included  in  this  paper.  In  the  management  information  system 
outlined  in  the  next  chapter  the  user  can  change  the  total  for  this  category,  but 
no  method  is  developed  here  for  investigating  what  changes  might  be  considered 
for  each  project  These  changes  would  undoubtedly  be  made  after  a  careful 
investigatioi't  of  the  organizational  slack  (fat)  in  each  of  Uie  headquarters  elements. 

H.  INVESTIGATING  THE  BUDGET  LEVEL  OF  THE  RDTE  PROGRAM 

A  characterization  of  an  optimal  solution  for  a  given  budget  constraint  (i.e.,  an 
optimal  mix  of  systems’  budgets  within  the  RDTE  program)  is  that  there  is  no 
other  way  that  the  funds  can  be  distributed  sucii  that  more  total  value  will  accrue 
to  the  program.  This  optimal  (or  efficient)  mbc  implies  nothing  about  what  the 
optimal  level,  or  total  RDTE  budget  constraint  should  be.  In  this  section  a  method 
is  developed  for  investigating  this  question. 

Suppose  that  over  the  past  several  years,  the  RDTE  budget  has  averaged  about 
nine  percent  of  the  total  Army  budget  and  that  under  current  DOD  fiscal  guidance, 
this  nine  percent  represents  1.7  billion  dollars.  Suppose  further  that  it  has  tentatively 
been  decided  to  fund  Categories  21  and  22  at  0.7  billion.  The  ques*  on  is  then. 
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should  the  remaining  budget  for  systems  be  one  billion,  a  little  more  or  a  little  less? 
Insight  into  this  question  might  be  gained  in  the  following  way. 

Using  Algorithm  2,  find  the  approximation  to  the  program  of  optimal  mix  for 
RDTE  budget  constraints  ranging  from  0.9  billion  to  1.1  billion,  in  increments  of 
10  million.  For  each  of  these  20  budget  constraints,  calculate 
n 

l  TV, 
i=l 

when  each  of  the  n  systems  are  funded  at  optimality.  Plotting  these  against  the 
20  budge;  constraint  points  might  lead  to  a  curve  similar  to  Figure  13.  If  such  a 


Figure  13.  TOTAL  PROGRAM  VALUE  VERSUS  TOTAL  COST  CURVE 

cur/e  were  to  result,  the  implication  is  that  increasing  the  budget  from  0.9  to 
1  billion  significantly  incieases  the  total  program  value,  while  a  relatively  smaller 
increase  is  achieved  when  going  from  1  to  1.1  billion. 

It  is  impossible  to  predict  vrithout  data  what  the  shape  of  this  curve  might 
be.  It  will  be  non-decreasing,  however,  since  it  results  from  the  addition  of 
systems’  value  curves  which  are  non-decreasing.  Whatever  its  exact  shape  turns  out 
to  be  will  determine  how  much  insight  into  the  problem  this  exercis'  might  provide. 


73 


For  example,  if  a  generally  linear  curve  were  to  result,  or  o^ic  where  no  discernable 
change  in  the  rate  of  increase  was  apparent,  relatively  little  insight  would  be 
provided  in  comparison  with  that  implied  by  the  Figure  13  curve.  In  fact  a 
linear  curve  would  provide  no  guidance  in  selecting  a  “proper”  budget  level,  since 
it  reflects  no  decrease  at  all  in  the  rate  at  which  total  program  value  increases  cs 
funds  are  increased. 
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V.  IMPLEMENTING  THE  MODEL  AS  A 


MANAGEMENT  INFORMATION  SYSTEM 

Although  the  solution  algorithms  for  this  model  tire  relatively  simple,  they 
require  a  great  deal  of  data  manipulation.  The  repetitious  nature  of  these  calculations 
lend  themselves  to  computerization,  and  the  model  itself  to  a  computer  assisted 
management  information  system  (MIS).  In  this  chapter  an  MIS  which  could  implement 
CARDBOMB  is  outlined;  one  which  seems  to  be  suitable  for  incorporation  into 
the  DEAN  MACHINE  system.  The  specific  desires  of  the  principal  users  would 
undoubtedly  result  in  an  MIS  quite  different  than  the  one  proposed  here.  This 
chapter  is  designed  more  to  point  out  the  potential  uses  of  the  model  in  assisting 
with  the  RDTE  budget  planning  problem,  and  the  relative  simplicity  of  the  computer 
programs  required.  It  might  also  prove  to  be  a  useful  starting  point  in  the  develop¬ 
ment  of  a  working  MIS. 

A,  THE  BACK-UP  SEGMENT  OF  THE  MANAGEMENT  INFORM.\TION  SYSTEM 

Unlike  pure  costing  models  which,  over  time,  tend  to  become  “believeable” 
to  users  when  their  time  saving  capabilities  and  accuracy  have  been  demonstrated, 
an  optimization  model  of  this  type  which  incorporates  many  factors  of  a  subjective 
nature  and  manipulates  them  m  an  almost  “magical”  fashion  has  to  be  clearly 
understood  to  be  useful.  The  amount  of  insight  into  a  decision  problem  that  a 
particular  solution  might  provide  is  directly  proportional  to  how  well  the  user 

OQ 

understands  the  model.  Consequently  an  MIS  must  consist  of  more  than  just  a 

oq 

For  a  discussion  of  some  of  the  pitfalls  of  designing  an  MIS,  o  Ackoff 
[ref.  m. 
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method  for  allowing  a  user  to  interface  with  a  computer.  A  users’  manual  which 


clearly  states  the  assumptions,  approximations,  methods,  capabilities  and  limitations 
of  the  model  is  an  integral  part  of  the  MIS.  Equally  importa/it  is  a  document  to 
which  the  user  can  refer  when  studying  the  parameters  of  systems  he  might  wish 
to  check  while  investigating  a  specific  solution  the  model  recommends.  Although 
OCRD  produces  a  Project  Listing  of  the  Program  6  base  case,  it  is  organized  in  the 
format  of  the  budget  and  recognizing  which  projects  belong  to  which  systems  is 
sometimes  a  difficult  task.  Additionally,  only  five  years  of  cost  information  is 
available  in  this  document.  The  reports  of  the  Military  Procurement  Priority 
Review  Board  have  systems’  priorities  and  costs,  but  generally  not  broken  down  by 
projects.  The  DCP’s  for  each  system  have  a  great  amount  of  information  for  the 
base  case,  but  this  model  also  requires  four  alternatives  to  the  base.  Consequently, 
should  CARDBOMB  be  implemented,  it  is  recommended  that  a  RDTE  System’s 
Data  Report  be  prepared  and  kept  current.  Besides  being  of  "reat  assistance 
to  users  of  tlio  model,  it  cculd  prove  to  be  an  excellent  planning  document.  A 
great  deal  of  cost  and  effectiveness  information  on  each  system  would  be  centralized, 
visible,  and  readily  available  for  five  alternative  development  plans.  Such  a  dccument 
and  the  model  itself,  would  no  doubt  be  most  efficiently  maintained  somewhere  in 
the  planning  staff  of  OCRD,  although  its  potential  users  might  be  per.sonnel  from 
the  offices  of  the  Director  of  the  Army  Budget,  the  Budget  Review  Conimittee,  and 
the  .Assistant  Vice  Chief  of  Staff,  in  addition  to  OCRD. 

B.  OPTIONAL  USER  INPUTS 

It  is  envisioned  that  this  MIS  wouid  operate  in  a  manner  siniUar  to  the  DEAN 
MACHINE  system;  i.e.,  the  user-computer  interface  would  be  through,  a  cathode  ray 
tuise  (CRT)  console.  The  user  types  in  simple  instiuctions  and  solutions  are  displayed 
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on  a  television  screen  before  him.  Some  of  these  instrud’-ons  are  presented  in  this 
section. 

Since  the  model  is  designed  to  handle  rltemative  metliods  for  altering  the  RDTE 
budget,  (e.g.,  alter  the  budget  constraint  and  do  not  consider  slipping,  fix  the 
constraint  and  consider  slipping,  alter  the  time  preference  strategy,  etc.)  the 
computer  program  segment  of  the  MIS  must  allow  for  the  input  of  thc^  options 
if  it  is  to  be  efficient  in  terms  of  running  time  and  storage  space.  Consequently, 
a  user  should  have  the  capability  of  inputing  uiy  oi  the  following  options. 

1.  Select  a  time  preference  strategy  The  most  time  consuming  operation  the 
program  might  be  required  to  accomplish  would  be  the  construction  of 
the  TVj  versus  cost  curves,  calculation  of  ..he  marginal  values  (slopes),  and 
the  ordering  of  these  slopes  in  decreasing  sequence.  It  might  therefore  be 
desireable  to  have  in  storage,  pre-constructed  budget  Lncrcmenting  matrices 
for  alternative  strategies.  A  great  deal  of  computer  time  could  be  saved 
this  way.  Therefore  the  following  time  preference  strategy  options  might 
be  made  aval'  ible. 

Option  1,  a  Short  Range  (a,  b,  c)  Strategy,  where  the  parameters  are  those 
determined  “best”  in  the  test  of  the  model  against  the  base  case  (Chapter  4, 
Section  C). 

Option  2,  Neutral  (1,  1,  1)  Strategy. 

Option  3,  Long  Range  (c,  b,  a)  Strategy,  using  the  same  parameters  as 
Option  1,  but  in  reverse  order 

Option  4,  User  selects  his  own  parameters.  As  has  been  staled,  the  analyst 
should  not  impose  a  strategy  on  the  decision  maker.  Option  4  allows  the 
user  to  input  his  own  time  preference  parameters,  but  he  should  be  made 
aware  of  the  fact  tl.it  the  solution  will  take  much  longer  lo  obtain. 

2.  Change  any  parameters?  .^s  familiarity  with  the  model  grows,  the  user 
may  decide  that  he  is  not  in  agreement  with  some  of  the  relative  value, 
readiness  d.'*te  distributions  or  other  parameters  on  file.  Alternatively 
he  may  wish  to  test  the  sensitivity  of  a  particular  solution  to  some  of 
the  value,  cost  or  other  parameters.  Here  he  is  given  the  option  to 
temporarily  change  any  of  the  parameters  he  wishes.  This  could  be 
accomplished  by  typing  a  code  number  (defined  in  Section  C)  to  identify 
the  parameters  and  the  new  value  he  desires  to  use. 

3.  Should  the  model  consider  slipping?  A  simple  yes-no  ans".  to  this  question 
would  be  sufficient  to  insure  that  the  proper  subroutines  were  used. 
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4.  What  fiscal  year  is  being  investigated?  To  answer  this  question  the  user 
types  in  the  year.  Although  the  development  of  the  model  dealt  primarily 
with  the. first  year  (FV  73),  any  budget  year  can  be  investigated.  Suppose 
for  a  particular  FY  73  constraint  the  optimal  solution  resulted  in  a  total 
FY  75  cost  that  was  considered  unreasonable.  A  constraint  could  then 

be  imposed  on  FY  75  and  the  funding  levels  of  systems  be  reduced  until 
this  new  constraint  is  met. 

5.  Select  the  total  RDTE  budget  constraint.  The  user  would  answer  this 
question  by  typing  in  the  dollar  constraint  he  wishes  to  investigate. 

6.  Select  the  constraint  for  basic  research.  If  this  were  the  same  as  for  the 
base  case,  the  model  would  by-pass  the  subroutine  designed  to  alter  the 
Category  22  budget.  If  it  were  different,  the  model  would  calculate  a 
new  Category  22  budget  according  to  the  decision  rules  discussed  in 
Chapter  4,  Section  G. 

7.  Select  the  constraint  for  mans^ement  and  svipport.  The  model  does 
nothing  with  this  other  than  to  subtract  if  from  the  total  constraint  to 
determine  how  much  is  left  for  the  remaining  systems.  Should  this 
constraint  be  less  than  the  base  case,  the  difference  would  represent  the 
user’s  personal  judgement  of  the  “organizational  slack”  that  can  be 
absorbed. 

8.  Select  the  outjjut  options.  At  this  point  the  user  informr.  the  computer 
program  what  he  desires  to  see  in  the  way  of  output.  Discussion  of  what 
these  options  might  b '  is  deferred  to  Section  D  of  this  chapter. 


C.  ORGANIZATION  OF  THE  DATA  INPUTS 

A  large  number  of  data  inputs  are  required  for  the  use  of  CARDBOMB.  How 

these  are  organized  can  have  a  significant  effect  on  the  efficiency  of  the  computer 

program.  Appendix  B  outlines  the  computer  program  and  subroutint-s  that  woula 

be  needed  for  implementing  the  MIS  in  the  fashion  proposed  in  this  chupter.  As 

ment’oned  in  the  discussion  on  inputing  strategy  ^ntions,  it  has  incorporated 

a  tradeoff  of  data  storage  for  reduced  running  time.  The  following  files  would 

be  required  for  operat'ng  the  MIS  in  the  manner  proposed  in  Appendix  B. 

File  I.  Budget  Incrementing  Matrix  (BIM)  for  strategy  option  1.  The  format 
for  this  pre-constructed  matrix  might  be  as  was  shown  in  Figure  7,  page 

File  2,  BIM  for  strategy  option  2.  Same  format. 

File  3,  BIM  for  strategy  option  3.  Same  format. 


78 


File  4.  Systems*  Parameter  File.  In  order  to  use  strategy  option  4  or  to  change 
some  of  the  parameters,  a  file  consisting  of  the  inputs  required  for  constructing 
TVj  versus  cost  curves  is  needed.  Of  course,  this  file  is  also  necessary  for 
constructing  Files  1,  2  and  3.  The  format  for  this  file  might  be  as  shown  in 
Figure  14. 
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Figure  14.  FORMAT  FOR  FILE  4 

Note  that  a  code  (ij)  would  be  sufficient  to  identify  the  parameter  that  a 
user  might  wish  to  change. 

File  5,  Basic  Research  Projects.  The  mathematical  operations  used  in  investigating 
alternative  Category  22  budgets  are  different  than  those  for  systems.  Grouping 
these  projects  separately  allows  for  the  separate  investigation  of  alternative  basic 
research  budgets.  The  format  for  File  5  could  take  the  form  of  that  shown  in 
Figure  15. 
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Figure  15.  FORMAT  FOR  FILE  5 
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File  6.  Yearly  Budget  Schedule.  It  may  have  been  noted  in  the  formats 
recommended  for  the  other  five  files  that  neither  annual  RDTE  nor  total 
PEMA  and  OMA  costs  for  systems  were  included.  This,  of  course,  could 
easily  have  been  done.  However  since  the  user  is  only  investigating  one  year 
at  a  time,  collecting  this  data  in  a  separate  file  can  assist  in  saving  core  storage 
space.  Figure  16  shows  the  format  this  file  might  take.  Note  that  if  the  user 
were  investigating  the  FY  73  budget,  only  the  columns  pertaining  to  that 
year  would  need  to  be  called  into  core  memory. 
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Figure  16.  FORMAT  FOR  FILE  6 

File  7,  Financial  Data  File.  Like  the  DEAN  MACHINE,  this  model  might  be 
called  upon  to  act  as  a  “report  generator”,  transforming  cost  data  from  one 
budget  format  to  another.  None  of  the  files  outlined  thus  far  contain  inform¬ 
ation  on  the  costs  (or  identity)  of  the  projects  which  compose  ilie  systems. 
This  information  could  be  included  in  Filt  7.  Although  Appendix  B  does 
not  outline  routines  for  incorporation  of  a  report  generation  capability,  this 
feature  could  be  made  a  part  of  CARDBOMB. 


D.  RELATING  COMPUTER  OUTPUTS  TO  INFORMATION  REQUIREMENTS 
There  are  a  number  of  capabilities  of  CARDBOMB  that  a  user  might  wish  to 
employ.  Allowing  him  to  select  an  output  option  insures  that  the  proper  subroutines 
are  called  and  the  particular  information  he  Lt  seeking  is  displayed.  Mow  much 
information  is  required  at  a  given  time  would  direct  whether  it  won :  l)e  best 
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displayed  on  the  CRT  or  on  a  computer  print-out.  The  following  list  of  output 
options  is  not  intended  to  be  exhaustive.  It  might  further  demonstrate  how  the  MIS 
could  be  used,  however. 

1.  CRT  display  of  changes  from  the  base  case.  Normally  a  user  would  wish 
to  focus  his  attention  on  the  changes  that  resulted  from  his  inputing  a 
new  parameter  or  budget  constraint.  This  could  be  accomplished  by  a 
visual  display  of  the  new  versus  base  funding  schedules  of  the  system  or 
basic  research  projects  which  changed. 

2.  CRT  display  of  changes  ^om  the  last  solution.  It  is  envisioned  that  the 
model  would  be  used  in  an  iterative  fashion,  i.e.,  once  a  solution  is  generated 
with  a  certain  set  of  parameters,  the  user  might  wish  employ  another 
option  to  see  how  it  affects  this  solution. 

3.  CRT  Summary  Display.  The  new  and  base  funding  schedules  aggregated  for 
each  of  the  22  categories  might  also  be  of  interest. 

4.  CRT  display  of  the  12  rows  of  the  Budget  Incrementing  Matrix  bracketing 
the  total  budget  constraint.  For  a  particular  budget  constraint  the  model 
recommends  change^  based  on  this  matrix.  The  user  might  also  wish  to 
see  what  other  changes  would  be  recommended  if  the  constraint  were 
slightly  greater  or  slightly  less.  Twelve  rows  are  selected  because  of  the 
12  line  output  capability  of  the  CRT. 

5.  Computer  print-cut  of  a  Budget  Incrementing  Matrix.  The  BIM  is  an 
ordered  list  of  increases  to  the  budgets  of  systems  as  the  total  budget 
constraint  is  increased.  Although  it  is  only  an  approximation  to  the 
optimal  solution,  it  could  be  used  as  a  readily  available  planning  document 
for  answering  the  “What  do  you  think  we  should  do  if  the  budget  is  cut” 
type  question. 

6.  Print-out  of  the  slip  sequence  list. 

7.  Print-out  of  the  dh'op  sequence  list. 

8.  Print-out  of  detailed  funding  schedules  of  all  systems  at  optimality. 

9.  Print-out  of  the  complete  RDTE  budget  in  alternative  formats. 
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VI.  SUMMARY 


Throughout  this  paper  an  attempt  has  been  made  to  point  out  both  the  capabil¬ 
ities  and  limitations  of  the  methods  used  in  the  model.  CARDBOMB  is  not  meant 
to  be  either  a  “cure-all”  for  the  RDTE  budgeting  problem  or  the  final  answer  for 
the  contribution  analytical  models  might  provide.  What  the  model  can  do  is 
provide  logical  consistency  between  information  gathered  in  the  relatively  “quiet” 
planning  phase  and  the  budget  decisions  made  during  the  “crunch”  period.  Even 
during  the  “crunch”  it  is  not  envisioned  that  the  solutions  generated  by  the  model 
should  be  arbitrarily  accepted.  However  the  budget  feasible  changes  it  recommends 
should  serve  to  narrow  the  focus  for  a  more  careful  investigation,  one;  which  might 
be  weU  served  by  the  comprehensive  data  bank. 

A.  CAPABILITIES  AND  LIMITATIONS  REVIEWED 

CARDBOMB  has  addressed  individually  the  entire  range  of  alternatives  that 
can  be  employed  for  altering  the  RDTE  budget.  These  were;  increase  or  decrease 
the  budgets  of  systems  or  projects,  slip  a  system  one  or  more  yc-ars,  add  or  drop 
a  system  or  project  from  the  program,  and  alter  the  RDTE  budget  total.  Various 
analytical  metliods  were  used  as  noted  below. 

Based  on  subjective  value  judgements  of  category  experts  and  cost  data  supplied 
by  program  managers,  value  versus  cost  curves  were  determined  for  each  system. 

Then  for  a  particular  budget  constraint,  an  algorithm  was  used  to  generate  an  approxi¬ 
mation  to  the  optimal  RDTE  budget.  In  this  wav  trade-offs  of  value  for  cost  were 
accomplished  for  each  system,  but  considered  as  a  part  of  the  total  'ogram,  not 


as  an  individual  system.  By  this  approximation  method  increases  or  decreases  to 
the  budgets  of  systems  are  recommended.  Basic  research  and  exploratory  development 
projects  are  recommended  for  reduction  or  increase  based  on  a  decision  rule  which 
insures  that  percentage  increases  (decreases)  are  applied  to  the  budget  of  projects 
which  are  directly  (inversely)  proportional  to  their  average  values. 

Dropping  a  system  or  project  is  also  handled  on  an  average  value  criteria. 

Because  the  model  treats  value  interdependently  and  PEMA  and  OMA  costs  as  negative 
value,  its  usefulness  for  assisting  with  this  decision  is  limited.  However  it  could  be 
used  to  focus  attention  for  further  investigation  on  those  systems  with  comparatively 
low  average  values. 

When  deciding  whether  or  not  a  proposed  system’s  development  plan  should 
be  undertaken,  the  model  can  assist  by  identifying  what  needs  to  be  given  up  in 
the  KDTE  program  in  terms  of  reduced  funding  levels  for  other  systems.  It  can  also 
help  to  identify  the  optimal  funding  level,  i.e.,  the  trade-off  of  cost  for  potential 
performance.  Additionally,  by  comparing  its  average  value  with  that  of  other  systems, 
insight  might  be  gained  on  whether  or  not  the  development  plan  should  be  undertaken. 

The  decision  on  whether  or  not  to  slip  a  system  is  handled  by  determining  a 
yearly  loss  in  value  that  would  result,  ai.d  recommending  those  systems  for  slipping 
whose  loss  in  value  is  less  than  the  gain  that  would  be  achieved  by  raising  the 
funding  levels  of  other  systems. 

It  is  hoped  that  the  model  could  add  insight  into  the  "proper”  total  budget 
constraint  by  plotting  a  total  value  versus  total  budget  curve  in  a  budget  interval 
around  the  historical  |>ercentage  of  the  total  Army  budget.  As  mentinnrxl  ir 
Chapter  4,  the  usefulness  of  this  method  is  dependent  on  the  shape  of  the  resulting 
curve. 
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B.  EXTENSIONS  OF  THE  MODEL 


Some  of  the  limitations  discussed  above  are  limitations  only  to  the  extent  that 
they  were  beyond  the  scope  of  this  paper.  The  interdependence  of  value  of  the 
systems  which  cause  the  model  to  be  limited  in  its  application  to  the  add  and  drop 
decisions  have  been  discussed  at  length  by  Weingartner  [ref.  9] ,  and  theoretical 
models  in  the  general  category  of  project  selection  are  available  which  could  be 
adapted  to  CARDBOMB.  The  treatment  of  costs,  however,  would  have  to  be 
handled  differently,  i.e.,  PEMA  and  OMA  uncouf*ed  from  the  measurement  of 
value.  This  would  suggest  that  perhaps  the  real  problem  is  not  the  optimization 
of  the  RDTE  budget  but  rather  the  optimization  of  the  RDTE,  PEMA  and  OMA 
budgets  combined.  Techniques  similar  to  those  used  in  this  paper  could  undoubtedly 
be  applied  in  most  cases  to  this  vastly  expanded  model. 

The  average  value  decision  rule  for  basic  research  projects,  though  intuitively 
appealing  since  it  says  to  add  the  most  money  to  those  which  are  expected  to 
return  the  most  value  per  dollar,  is  not  theoretically  sound.  The  appropriate 
measurement  for  incremental  changes  in  budgets  is  marginal  value,  not  average 
value.  A  more  thorough  examination  of  Categoi-y  22  projects  might  suggest  a 
more  appropriate  structure  for  mvoueling  Basic  Research. 

Of  course  a  very  important  extension  of  the  model  would  oe  to  implement  it 
and  see  if  it  is  useful  in  assisting  in  the  RDTE  budgeting  problem.  This  would  be 
a  sizeable  task  th’*  would  no  doubt  be  best  accomplished  in  stages.  For  example 
very  little  would  need  to  be  changed  in  the  computer  programs  for  the  final  model 
if  they  were  first  written  to  accommodate  tin?  systems  from  only  one  category. 
Perhaps  Category  5,  STANDS,  might  be  a  suiU\ble  starting  point  since  all  these 


84 


lystems  come  under  a  single  program  manager.  Since  the  data  manipulations  for 
these  few  systems  would  not  be  excessive,  the  model  could  be  tested  without  the  use 
of  the  computer. 
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APPENDIX  A 

MATHEMATICAL  CONCEPTS 


This  appendix  is  an  extension  of  some  of  the  concepts  discussed  in  the  main  body 
of  the  paper.  Details  concerning  these  concepts  were  purposely  omitted  from  the  body 
of  the  text  so  that  it  would  be  more  easily  read  by  individuals  not  thoroughly 
familiar  with  the  jargon  of  Operations  Research.  For  those  readers  with  an  OR 
background,  this  appendix  should  assist  in  clarifying  two  important  questions;  what 
is  embodied  in  the  value  parameters,  and  what  is  the  formulation  of  the  optimization 
program. 

The  concept  of  relative  value  as  used  in  this  paper  is  similar  to  that  found  in 
Fishbum  [lef.  12].  Let  V  be  a  value  function  which  maps  a  real  number  V(s: :)  to 
a  system  Sjj  from  the  set  of  systems  Sj  in  tlie  category.  The  survey  respondent 
accomplishes  this  mapping  considering 

1.  Sj 

2.  Nj;  the  set  of  all  perceived  needs  for  systems  in  the  category,  where 

Nj  is  a  function  of, 

3.  E;  the  environment  —  his  perception  of  the  enemy  threat,  the  range 

of  possible  scenarios,  etc. 

That  is: 

V(sy)  -  Visy;  Sj,  Nj(E)] 

With  the  usual  convention  tliat  if  s^y  is  considered  more  valuable  than  sj^j  then 
V(sjj)  >  V(S)fj),  the  optimization  methods  used  require  that  V  be  unique  up  to 
an  increasing  linear  transformation.  In  other  words,  the  results  of  the  survey  on 
the  zero-ten  scale  must  be  considered  to  have  established  an  interval  ^  ale  of 
measurement,  meaning  that  shifting  the  origin  of  the  scale  or  multiplying  it  by  a 
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positive  constant  will  leave  the  relative  lengths  of  the  intervals  between  V(sy) 
unchanged.  For  example: 

V(Sij)  -  V(skj)  aV(8y)  +  b  -  (aV(s;y)  +  b]  ^ 

V(»lj)  •  V(s„j)  "  aV(y)  +  b  -  [aV(s„j)  +  b]  ‘  ° 

This  requirement  for  an  interval  scale  of  measurement  could  be  tested  by  surveying 
again  to  confirm  that  the  zero-ten  scale  produces  the  same  relative  interval  results 
as,  for  example,  a  zero-one  hundred  scale. 

When  the  primary  method  (see  page  is  used  to  relate  the  value  of  each 
system  to  that  of  a  program  numeraire,  the  high-level  DA  managers  are  asked  to 
use  the  “ranking-rating”  method  for  each  of  the  category  numeraires.  Thus  they 
are  being  asked  to  map  real  numbers  to  the  values  of  these  systems  considering: 

1.  Sj,  .  .  .  ,  S2q;  the  sets  of  systems  in  each  category. 

2.  S';  the  set  of  numeraire  systems. 

3.  Nj,  .  .  .  ,  N20;  the  set  of  needs  in  each  category. 

4.  E;  the  environment 
That  is: 

V(Sqj)  =  V[sqj;  Sj,  .  .  .  ,  S20.  S'.  N^fE) . NgoiE)! 

Using  the  primary  method  for  relating  the  value  parameters  of  all  the  systems  requires 
that  the  mappirig  V(Sqj)  should  also  result  in  an  interval  scale  of  rnc.-rsurement.  The 
use  of  equation  2,  page  29,  is  equivalent  to  finding  “a”  in  the  linear  transformation, 
i.e.,  determining  the  constant 

^q.i 

which  compresses  or  stretches  the  value  scale  of  the  category  to  ■  ake  it 
commensurate  with  the  scale  of  the  program  numeraire. 
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The  secondary  method,  page  30,  also  seeks  to  determine 


but  in  a  different  manner.  To  use  equation  4,  page  32, 


must  be  determined  by  survey.  The  high-level  managers  must  consider: 

1.  Nj,  .  .  .  ,  N20 

2.  E 

The  value  function  mapping  is  then: 

V(Nj)  =  V[Nj;Ni(E),....N2o(E)l 

All  the  value  measurement  methods  discussed  in  the  text  also  require  the  additivity 
assumption,  i.e., 

^(<*1  j  +  S2j  +  .  ,  .  +  Sy)  -  V(si j)  +  V(s2j)  +  .  .  .  +  V(sy)  for  i»l,  .  .  .,  nj 
For  this  assumption  to  hold,  systems  must  be  carefully  defined.  For  example,  a 
weapon  and  its  platform  must  be  one  system.  The  assumption  could  be  tested 
(with  great  effort)  with  surveys  designed  along  the  lines  of  the  Churchman-Ackoff 
method  for  obtaining  measures  of  relative  value  (see  Burington  [ref.  6,  p.  26]).  In 
brief,  this  would  require  that  each  pair  of  systems,  then  each  triplet,  etc.,  be  tested 
by  survey  to  insure  that  the  additivity  assumption  holds. 

Without  any  testing  of  the  parameters  obtained  by  the  “ranking-rating”  method, 
there  is  no  way  of  insuring  that  consistent  judgement  has  been  applied  in  formulating 
the  value  function,  V.  In  other  words  it  must  be  presumed  that  the  usual  axioms 
for  the  existence  of  a  real  valued  utility  function  held,  i.e.,  that  the  reference 
relation  is  reflexive,  symetric  and  transitive.  (Sec,  for  e.\ample,  Fi:>i.  .rn  (ref.  12, 
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p.  167]).  The  testing  required  to  insure  that  the  axioms  hold  would  result  in  a  very 
sizeable  surveying  effort,  one  which  is  felt  by  the  author  to  be  an  unreasonable 
undertaking  in  practice  due  to  the  time  that  might  be  allotted.  Eckenrode  [ref.  13] 
has  shown  in  a  limited  test  of  the  rating  method  that  no  statistically  significant 
differences  in  results  were  obtained  from  this  method  and  some  of  the  more  complex 
ones  which  do  test  for  compliance  with  the  axioms.  Of  course  his  study  has  only 
limited  relevance  to  this  particular  problem.  Consequently,  the  parameters  which 
result  from  the  relatively  simple  methods  outlined  in  the  body  of  the  text  should 
not  be  viewed  as  being  perfectly  coirelated  with  some  “actual”  real  valued 
utility  function  for  the  Department  of  the  Army.  The  sensitivity  of  a  solution 
to  the  relative  value  parameters  used  should  always  be  checked. 

Algorithm  1,  page  SS,  results  from  the  formulation  of  the  problem  as  a 
mathematical  program  in  the  following  way. 

-A  0  0  0 

We  wish  to  find  the  optimal  funding  level  f^'  =  (fj^,  f2,  .  .  .  ,  fj,)  which 
maximimizes  the  total  value  of  the  program  and  satisfies  the  budget  constraint  for 
the  year  being  investigated.  That  is: 

Maximize:  g(  f ) 

Subject  to:  h(f)<B^ 

For  the  i^^  system,  the  problem  has  been  formulated  as: 

TVi(Xi)  -  •  at  •  Pt(Xi)  •  Vj(Xj  :  Sj.  Nj(E)] 

Although  a  separable  programming  approach  is  not  the  only  formulation  that  might 
be  considered,  it  does  result  in  the  simplest  solution  methods.  For  g(  f )  to  be  a 
separable  function  it  is  necessary  to  assume  that: 

VjlXi  ,  Sy.  S^.  Nj(E)l  -  Vj(Xi) 
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that  is,  the  value  of  the  system  is  independent  of  the  values  of  all  the  others.  As 
discussed  in  the  text,  it  has  been  assumed  that  in  the  funding  range  [f  2, .  .  .  ,  f  -  6] 
this  independence  of  systems*  value  holds.  “Very  often  the  choice  between  independ¬ 
ence  or  not  is  a  choice  between  ‘divide  and  conquer  with  approximations*  or  ‘don’t 
divide  and  don’t  conquer  at  all.’  ’’  Fishburn  [ref.  l2,  p.  295].  However  “don’t 
conquer  at  all’’  is  not  quite  the  case  with  this  problem.  As  has  been  stated, 

Weingartner  [ref.  9]  discusses  a  number  of  models  that  take  into  account  the 
interdependence  of  value.  Some  of  them  would  be  appropriate  for  the  “add”  and 
“drop”  decisioti  and  could  be  adapted  to  CARDBOMB. 

With  the  assumption  of  independence  of  value,  the  program  becomes: 

Maximize:  g(f)  “  Z  TVj(xj) 

i 

Subject  to:  h(  f )  .< 

The  budget  constraint  equation  is  logically  separable  into  the  year  t  budgets 

for  each  of  the  systems,  i.e.,  h(f)  C-  ^(fj)  <  B^,  where  Cj^{fj)  is  a  discrete  function 

i 

in  f,  the  five  finding  levels.  The  final  mathematical  program  is  then: 

Maximize:  ^  TVj(xj) 

i 

Subject  to:  ^  ^it^^i^  —  ®t 

I 

fj  •  fj  for  k  =  2,  or  3  ...  or  6 


Algorithm  1  accomplishes  this  optimization  by  first  calculating  the  marginal  values, 
sj  jj,  of  raising  the  funding  level  from  f“ktof“k  +  l;  i.e., 

TVi(fi.k.l)  TVi(fi^k) 
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When  the  set  of  S|]^  are  ordered  in  decreasing  sequence,  iteratively  increasing  the 
funding  levels  according  to  this  sequence  insures  that  the  increase  has  been  added 
to  the  system  with  the  highest  gain  in  value  per  total  RDTE  dollar  increase.  However 
as  was  pointed  out  on  page  54,  this  is  not  necessarily  the  optimal  increase  when  a 
year  t  budget  constraint  is  considered.  Lacking  information  on  these  year  t 
constraints  for  the  out-years,  the  question  becomes  one  of  choosing  a  criteria; 
either  maximize  value  considering  the  marginal  value  of  total  cost  or  the  marginal 
value  of  year  t  cost.  Of  course  Algorithm  1  uses  the  former  criteria,  but  a  user 
of  the  model  might  also  wish  to  investigate  a  solution  resulting  from  the  latter.  The 
two  solutions  would  be  identical  if  the  percentage  differences  in  total  cost  between 
each  funding  level  were  the  same  as  the  percentage  differences  in  year  t  costs  for 
all  systems.  In  this  case  the  sequence  of  Sj  would  be  the  same  as  a  sequence 
resulting  from  using  f+i  t  ‘  ^i  f  t  denommator  of  the  formula. 

A  number  of  nonlinear  programming  search  techniques  might  be  applicable 
to  finding  the  global  optimum  for  a  particular  year  t  budget  constraint.  Unfortunately 
until  such  time  as  data  might  be  collected  and  the  shape  of  the  TVj  versus  year  t 
cost  curves  determined,  it  is  impossible  to  predict  how  divergent  the  two  solutions 
would  be,  or  to  recommend  a  particular  solution  algorithm.  Of  course  if  these  curves 
were  also  cone  ve  in  the  region  of  positive  value,  using  a  sequence  of  marginal  values 
of  year  t  cost  in  Algorithm  1  would  be  a  simple  method. 
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APPENDIX  B 
CONCEPT  FLOW  CHART 


This  appendix  is  added  to  demonstrate  the  reasonably  simple  nature  of  the 
computer  program  that  would  be  required  to  implement  CARDBOMB.  It  is  designed 
to  give  the  reader  a  better  understand  of  the  MIS  outlined  in  Chapter  5,  not  to 
present  a  detailed  programming  flow  chart.  Readers  with  some  familiarity  with 
programming  may  note  that  the  computer  operations  required  are  neither  complex 
or  excessively  time  consuming. 
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Using  (f.TVjf) 
Calculate  Slopes 


Oider  the  Slopes 
Into  a  Budget 
Increnu  nting 
Matm 


Figure  18.  STRATEGY  OPTION  4  SUBROUTINE 


Figure  19.  PARAMETER  CHANGE  SUBROUTINE 
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Figure  20.  BASIC  RESEARCH  SUBROUTINE 
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